Choosing F-Test Denominators for Split-Plot Effects

Why the Denominator Depends on the Design

In a split-plot experiment, treatments are assigned at two different levels. A whole-plot treatment is assigned to an entire whole plot, while a subplot treatment is randomized among subplots inside each whole plot. The observations therefore do not provide the same kind of replication for both treatments. In particular, subplots receiving different whole-plot treatments are not independent replications of those whole-plot treatments.

This difference determines how treatment variation should be compared with random variation. The whole-plot treatment is tested against variation among whole plots given the same treatment. The subplot treatment and the whole-plot-by-subplot interaction are tested against variation among subplots within whole plots. Choosing the denominator is not a matter of selecting whichever mean square looks convenient: it follows from the randomization structure and the expected mean squares.

Definition: An error stratum is a source of random variation associated with a particular level of experimental units. In a balanced split-plot design, the whole-plot error stratum measures variation among whole plots, and the subplot error stratum measures variation among subplots within whole plots. A valid F test uses a denominator mean square from the error stratum appropriate to the effect being tested.

Model and the Three Denominator Choices

Consider a balanced split-plot design with \(r\) blocks, \(a\) levels of whole-plot treatment \(A\), and \(b\) levels of subplot treatment \(B\). Each block contains one whole plot at each level of \(A\), and each whole plot contains one subplot at each level of \(B\). Write \(Y_{ijk}\) for the response in block \(i\), whole-plot treatment group \(j\), and subplot treatment group \(k\).

Y_{ijk}=\mu+\rho_i+\alpha_j+u_{ij}+\beta_k+(\alpha\beta)_{jk}+\varepsilon_{ijk}.

Here \(\mu\) is the overall mean; \(\rho_i\) is the block effect; \(\alpha_j\) and \(\beta_k\) are the fixed treatment effects; \(u_{ij}\) is the random whole-plot error; and \(\varepsilon_{ijk}\) is the random subplot error. Assume independent, normally distributed random effects and errors, with \(u_{ij}\sim N(0,\sigma_W^2)\) and \(\varepsilon_{ijk}\sim N(0,\sigma_S^2)\). The whole-plot and subplot variances are constant across their respective units. Treatments are assigned according to the stated split-plot randomization, and the design is balanced.

The expected mean squares show which denominator makes the null hypothesis give an F ratio with a central F distribution. For the treatment effects and error terms, the relevant results are:

\begin{aligned} E(MS_{A})&=\sigma_S^2+b\sigma_W^2+ \frac{rb\sum_{j=1}^{a}\alpha_j^2}{a-1},\\ E(MS_{\text{whole-plot error}})&=\sigma_S^2+b\sigma_W^2,\\ E(MS_{B})&=\sigma_S^2+ \frac{ra\sum_{k=1}^{b}\beta_k^2}{b-1},\\ E(MS_{A\times B})&=\sigma_S^2+ \frac{r\sum_{j=1}^{a}\sum_{k=1}^{b}(\alpha\beta)_{jk}^{2}} {(a-1)(b-1)},\\ E(MS_{\text{subplot error}})&=\sigma_S^2. \end{aligned}

Under \(H_0\) for treatment \(A\), all \(\alpha_j=0\), so the treatment contribution to \(E(MS_A)\) vanishes. Its expected mean square then matches the whole-plot error mean square. Under \(H_0\) for \(B\), the contribution from the \(\beta_k\) values vanishes, and \(MS_B\) matches the subplot error mean square. Under the no-interaction null hypothesis, the interaction contribution vanishes, and \(MS_{A\times B}\) also matches the subplot error mean square.

Key result: Test \(A\) using \(MS_A/MS_{\text{whole-plot error}}\). Test \(B\) using \(MS_B/MS_{\text{subplot error}}\). Test \(A\times B\) using \(MS_{A\times B}/MS_{\text{subplot error}}\). The numerator and denominator degrees of freedom are, respectively, \(a-1\) and \((r-1)(a-1)\) for \(A\); \(b-1\) and \(a(r-1)(b-1)\) for \(B\); and \((a-1)(b-1)\) and \(a(r-1)(b-1)\) for the interaction.

Why the Whole-Plot Error Matches the Whole-Plot Treatment

The whole-plot treatment is assigned to whole plots, not separately to each of their subplots. A useful way to see the consequence is to average the \(b\) subplot responses in a whole plot. Under the model, the average contains the whole-plot error \(u_{ij}\) unchanged, while averaging the \(b\) independent subplot errors reduces their variance from \(\sigma_S^2\) to \(\sigma_S^2/b\). Thus, the variance of a whole-plot average due to these two error sources is \(\sigma_W^2+\sigma_S^2/b\). Multiplying by \(b\) gives the expected mean-square scale \(\sigma_S^2+b\sigma_W^2\) for the whole-plot comparison.

Under the null hypothesis for \(A\), both the whole-plot treatment comparison and the whole-plot error estimate contain that same variance scale. Their ratio is therefore the appropriate F statistic. The subplot error mean square estimates only \(\sigma_S^2\); it does not include \(b\sigma_W^2\). If whole-plot variability is positive, dividing \(MS_A\) by the subplot error can make the denominator too small and the F statistic too large.

Worked Example: Matching Each Effect to Its Error Stratum

A hypothetical experiment uses four blocks, three levels of a whole-plot treatment \(A\), and two levels of a subplot treatment \(B\). Identify the denominator and degrees of freedom for each treatment test.

Here \(r=4\), \(a=3\), and \(b=2\). The whole-plot error degrees of freedom are \((r-1)(a-1)=(4-1)(3-1)=6\). The subplot error degrees of freedom are \(a(r-1)(b-1)=3(4-1)(2-1)=9\).

For \(A\), the numerator degrees of freedom are \(a-1=2\), and the denominator is the whole-plot error with 6 degrees of freedom. Its test is \(F_A=MS_A/MS_{\text{whole-plot error}}\), compared with an \(F\) distribution with 2 and 6 degrees of freedom under \(H_0\).

For \(B\), the numerator degrees of freedom are \(b-1=1\), and the denominator is the subplot error with 9 degrees of freedom. Its null reference distribution is \(F\) with 1 and 9 degrees of freedom.

For \(A\times B\), the numerator degrees of freedom are \((a-1)(b-1)=2(1)=2\), and the denominator is again the subplot error with 9 degrees of freedom. Its null reference distribution is \(F\) with 2 and 9 degrees of freedom.

The two subplot-related tests share a denominator because both are compared with variation among subplots after accounting for the treatment effects. They do not use the whole-plot error merely because their observations happen to be grouped within whole plots.

Calculating the F Statistics

For each effect, divide its mean square by the mean square from its designated error stratum. A mean square is its sum of squares divided by its degrees of freedom. The F statistic is dimensionless: its numerator and denominator are both measured in squared response units.

Worked Example: Three F Tests From One Balanced ANOVA

Suppose a hypothetical balanced experiment has \(r=5\) blocks, \(a=3\) levels of \(A\), and \(b=2\) levels of \(B\). Its sums of squares are \(SS_A=80\), \(SS_{\text{whole-plot error}}=64\), \(SS_B=30\), \(SS_{A\times B}=18\), and \(SS_{\text{subplot error}}=48\). Calculate the three F statistics and their degrees of freedom.

For \(A\), the degrees of freedom are \(2\) for treatment and \((5-1)(3-1)=8\) for whole-plot error. Therefore,

MS_A=\frac{80}{2}=40,\qquad MS_{\text{whole-plot error}}=\frac{64}{8}=8,\qquad F_A=\frac{40}{8}=5.00.

The \(A\) test uses an \(F\) distribution with 2 and 8 degrees of freedom under its null hypothesis.

For \(B\), the treatment degrees of freedom are \(2-1=1\), while the subplot error degrees of freedom are \(3(5-1)(2-1)=12\). Thus,

MS_B=\frac{30}{1}=30,\qquad MS_{\text{subplot error}}=\frac{48}{12}=4,\qquad F_B=\frac{30}{4}=7.50.

The \(B\) test uses an \(F\) distribution with 1 and 12 degrees of freedom under \(H_0\).

For the interaction, the numerator degrees of freedom are \((3-1)(2-1)=2\), and the denominator degrees of freedom are again 12. The calculation is

MS_{A\times B}=\frac{18}{2}=9,\qquad F_{A\times B}=\frac{9}{4}=2.25.

The interaction test uses an \(F\) distribution with 2 and 12 degrees of freedom under the no-interaction null hypothesis. Each statistic is compared with the upper tail of its own reference distribution, using its own numerator and denominator degrees of freedom. A conclusion about statistical significance requires a specified significance level \(\alpha\) or a corresponding p-value; the F statistics alone do not establish that a treatment effect is practically important.

A Short Justification From the Expected Mean Squares

The denominator choices can be justified by checking the null expected mean squares directly. For the whole-plot treatment, set all \(\alpha_j=0\). The treatment term in \(E(MS_A)\) is then zero, leaving

E(MS_A\mid H_0)=\sigma_S^2+b\sigma_W^2 =E(MS_{\text{whole-plot error}}).

For the subplot treatment, setting all \(\beta_k=0\) removes its treatment contribution and gives

E(MS_B\mid H_0)=\sigma_S^2 =E(MS_{\text{subplot error}}).

Similarly, under the no-interaction null hypothesis, every \((\alpha\beta)_{jk}=0\), so

E(MS_{A\times B}\mid H_0)=\sigma_S^2 =E(MS_{\text{subplot error}}).

These equalities explain why the specified ratios have central F reference distributions under their respective null hypotheses, given the model assumptions and balanced design. The match is specific to each null hypothesis: the whole-plot treatment includes a whole-plot variance component, while the two subplot-level effects do not.

Worked Example: Diagnosing a Wrong Denominator

In a hypothetical experiment, \(MS_A=36\), \(MS_{\text{whole-plot error}}=9\), and \(MS_{\text{subplot error}}=3\). A student tests \(A\) by dividing by the subplot error. Show why that choice changes the statistic and identify the valid one.

Because \(A\) is assigned to whole plots, its correct denominator is the whole-plot error. The valid statistic is

F_A=\frac{MS_A}{MS_{\text{whole-plot error}}} =\frac{36}{9}=4.00.

Using the subplot error instead gives \(36/3=12.0\). This is not an alternative calculation of the same test: it compares the whole-plot treatment mean square with a different variance scale. Under the null hypothesis for \(A\), its expected mean square includes \(\sigma_S^2+b\sigma_W^2\), whereas the subplot error estimates \(\sigma_S^2\). If \(\sigma_W^2>0\), these expected values differ, so the ratio does not have the intended central F reference distribution.

The larger value of 12.0 may appear to provide stronger evidence against \(H_0\), but that appearance is produced by a mismatched denominator. A valid p-value must come from the statistic and reference distribution justified by the design.

When the Match Needs Reconsideration

The stated denominator rules apply to the balanced design and model specified here. They rely on the stated randomization, independent random effects and errors, common variance within each error stratum, and the expected mean squares shown above. If the design is unbalanced, observations are missing, or the random-effects structure differs, the simple mean-square ratios may not have the stated reference distributions. The error term should then be derived from the model actually used rather than selected by analogy alone.

The labels “whole-plot error” and “subplot error” refer to sources of variation, not necessarily to a single universal physical cause. Their precise composition depends on how the experiment was randomized and modeled. What matters for denominator selection is whether the denominator mean square has the same expected value as the numerator mean square under the null being tested.

Common Mistake

Wrong reasoning: “All observations are measured on the same response scale, so one pooled residual mean square can be used for every treatment.”
Correction: A shared response scale does not make the randomization units interchangeable. The whole-plot treatment must be tested against whole-plot variation; the subplot treatment and interaction use the subplot error in this balanced model. Check the expected mean squares under the relevant null hypothesis before forming an F ratio.
Another warning: The number of recorded subplots is not the number of independent whole-plot treatment assignments. Treating subplots as independent replications for \(A\) ignores the whole-plot error stratum.

Key takeaway: Match the denominator to the level at which the effect was randomized and verify the match through the expected mean squares. In the balanced split-plot model here, \(A\) uses whole-plot error, while \(B\) and \(A\times B\) use subplot error.

Check Your Understanding

For each question, name the relevant error stratum and explain the denominator choice.

In a balanced split-plot design with \(r=4\), \(a=2\), and \(b=3\), what are the denominator degrees of freedom for the \(A\) test and the \(B\) test?
Why does the whole-plot treatment test not use the subplot error mean square in the model given here?
A student calculates \(F_{A\times B}\) using the whole-plot error denominator. What is the conceptual error in that choice?
Under the null hypothesis for \(A\), what variance components appear in the expected mean square for \(A\), and which error mean square has the same expectation?
If the design is unbalanced, is it automatically valid to keep using the same three mean-square ratios? Explain what must be checked.