Why Careful Table Reading Matters
A balanced two-factor ANOVA table compresses several calculations into a small space. That makes it efficient, but also makes errors easy to miss: a degree of freedom can be assigned to the wrong row, a sum of squares can be mistaken for a mean square, or a main-effect test can be described as though it compared every pair of factor levels. A reliable reading starts by checking the model and the table’s arithmetic before interpreting a test.
This tutorial focuses on common errors in balanced, fixed-effects, two-factor ANOVA. The checks apply whether the table was calculated by hand or produced by software. They do not replace examining the design and assumptions; a perfectly consistent table can still be inappropriate if the model does not fit the data.
The Model and the Table’s Basic Structure
Suppose factor A has \(a\) levels, factor B has \(b\) levels, and the design has \(n\) observations in every A-by-B cell. For observation \(k\) in cell \((i,j)\), the fixed-effects model is
Here \(\mu\) is the overall mean, \(\alpha_i\) and \(\beta_j\) are the factor effects, and \((\alpha\beta)_{ij}\) is the interaction effect. The effects use identifying constraints, such as sum-to-zero constraints. The errors are assumed independent, normally distributed with mean 0 and common variance \(\sigma^2\). The response is quantitative, and observations are obtained under a design in which the independent-error assumption is reasonable. The equal cell size \(n\) is what makes this a balanced design.
The sums of squares partition the corrected total variation into variation attributable to A, B, their interaction, and within-cell error:
The corresponding degrees of freedom are \(a-1\), \(b-1\), \((a-1)(b-1)\), and \(ab(n-1)\), respectively. The corrected total has \(abn-1\) degrees of freedom. For each row, the mean square is its sum of squares divided by its degrees of freedom. For testing a main effect in this fixed-effects model, the F ratio is the effect’s mean square divided by the error mean square.
A Short Justification for the Checks
The degrees-of-freedom check follows from counting independent pieces of information. There are \(abn\) observations, so deviations from the estimated grand mean have \(abn-1\) degrees of freedom. Within a cell, \(n\) observations provide \(n-1\) independent deviations after estimating that cell’s mean. Across \(ab\) cells, this gives \(ab(n-1)\) error degrees of freedom. The remaining degrees of freedom describe the variation among cell means: A uses \(a-1\), B uses \(b-1\), and the interaction uses the residual \((a-1)(b-1)\).
Adding the components confirms the count:
The same partition applies to sums of squares in a balanced design. This gives two quick table checks: the effect and error degrees of freedom must add to the total, and the component sums of squares must add to the total sum of squares. A discrepancy is a signal to investigate, not something to fix by changing a displayed number without checking the calculations.
Worked Example: Checking a Main-Effect Test
Worked Example: Checking a Main-Effect Test
A hypothetical balanced experiment has two levels of A, three levels of B, and four observations in each cell. The reported sums of squares are \(SS_A=96\), \(SS_B=24\), \(SS_{AB}=12\), and \(SS_E=144\). Check the A test and the table’s overall degrees of freedom.
Check the design and degrees of freedom. There are \(abn=2(3)(4)=24\) observations. The degrees of freedom are \(df_A=2-1=1\), \(df_B=3-1=2\), \(df_{AB}=(2-1)(3-1)=2\), and \(df_E=2(3)(4-1)=18\). They sum to \(1+2+2+18=23\), which equals \(24-1\), the total degrees of freedom.
Check the sums of squares. The four components sum to \(96+24+12+144=276\). Thus the corrected total sum of squares must be 276 for this table to be coherent.
Compute the mean squares and A ratio. The error mean square is \(MS_E=144/18=8\), while \(MS_A=96/1=96\). Therefore,
Under \(H_0\) of no A main effect, with the stated model assumptions, this has an \(F\) distribution with 1 and 18 degrees of freedom. Its upper-tail p-value is approximately 0.002769. At a significance level of 0.05, the result is statistically significant evidence against \(H_0\). In context, the test concerns equality of the two A marginal means, averaged over the three B levels; it is not a separate test for every cell or every possible comparison.
Common Numerical Errors
The table below summarizes frequent misreadings and the check that corrects each one.
| Misreading | Correction |
|---|---|
| Using a sum of squares directly as an F numerator | Divide the sum of squares by its degrees of freedom to get the mean square first. |
| Dividing an effect mean square by the interaction mean square for a main-effect test | In the fixed-effects model considered here, the main-effect F ratio uses \(MS_E\) as its denominator. |
| Adding only the three model rows’ degrees of freedom | Include error degrees of freedom; all components must add to the corrected total degrees of freedom. |
| Reading a p-value as a lower-tail probability | ANOVA F tests use the upper tail: larger F statistics provide more evidence against the null hypothesis. |
| Calling a significant main effect proof that every level differs | A main-effect test concerns marginal means. It does not by itself establish which levels differ or that all differences are nonzero. |
A further trap is assuming that a small p-value measures the size or practical importance of an effect. The p-value describes how unusual an F statistic at least as large as the observed statistic would be under the null hypothesis and the model assumptions. It is not the probability that the null hypothesis is true, nor is it a measure of effect size.
Worked Example: Using the Correct F Tail and Degrees of Freedom
Worked Example: Using the Correct F Tail and Degrees of Freedom
A second hypothetical balanced experiment has two levels of A, two levels of B, and seven observations per cell. Its reported A row has \(SS_A=18\), and its error row has \(SS_E=96\). Check the A test and its p-value.
There are \(2(2)(7)=28\) observations. Thus \(df_A=1\), \(df_B=1\), \(df_{AB}=1\), and \(df_E=2(2)(7-1)=24\). The total degrees of freedom are \(27\), which agrees with \(28-1\). The error mean square is \(MS_E=96/24=4\), and the A mean square is \(MS_A=18/1=18\). Hence,
The p-value is the upper-tail probability for an \(F\) distribution with 1 and 24 degrees of freedom: \(P(F_{1,24}\geq 4.5)\approx 0.04442\). Since this is below 0.05, the test is significant at that level. Reporting a p-value such as 0.0441 would not be a suitable rounding of this calculation; to five significant figures the value is about 0.04442.
The F distribution has only nonnegative values, and this test looks for large F statistics. Using a lower-tail probability, or using a different denominator degree of freedom, would answer the wrong question. The degrees of freedom belong to the test distribution: here they come from the A row and the error row, not from the corrected total row.
What a Marginal Main-Effect Test Says
In a balanced design, a main-effect comparison averages equally across the levels of the other factor. If \(\bar{Y}_{i..}\) is the sample mean for A level \(i\), averaged over all B levels and observations, the A main-effect null hypothesis is that all \(a\) A marginal means are equal. The B test is defined in the corresponding way using B marginal means. For a factor with two levels, its main-effect test is a test of equality of those two marginal means.
This averaging matters when explaining a result. A significant A test does not mean that every A-level pair is different when A has more than two levels. It also does not mean that the A difference has the same size at each B level. Conversely, a nonsignificant test does not prove that the marginal means are exactly equal; it says the data do not provide sufficient evidence against their equality at the chosen significance level. The interaction row is a separate part of the table and addresses a different null hypothesis.
Worked Example: Auditing the Whole Table
Worked Example: Auditing the Whole Table
A hypothetical balanced design has three A levels, two B levels, and five observations in every cell. A table reports the following sums of squares: \(SS_A=40\), \(SS_B=18\), \(SS_{AB}=12\), and \(SS_E=96\). Audit the degrees of freedom, mean squares, and main-effect F ratios.
Degrees of freedom. There are \(3(2)(5)=30\) observations, so total \(df=29\). The component degrees of freedom are \(df_A=3-1=2\), \(df_B=2-1=1\), \(df_{AB}=(3-1)(2-1)=2\), and \(df_E=3(2)(5-1)=24\). Their sum is \(2+1+2+24=29\), as required.
Sums of squares. The reported components sum to \(40+18+12+96=166\), so the corrected total sum of squares should be 166.
Mean squares and tests. The error mean square is \(MS_E=96/24=4\). Then \(MS_A=40/2=20\), so \(F_A=20/4=5\). Also, \(MS_B=18/1=18\), so \(F_B=18/4=4.5\). The table’s main-effect tests therefore use numerator degrees of freedom 2 and 1, respectively, and denominator degrees of freedom 24 for both. A report that gives \(SS_A=40\) as the A mean square, or uses \(MS_{AB}=12/2=6\) as the denominator for either main-effect test, has made a calculation error.
Because A has three levels, its test evaluates equality of three A marginal means; \(F_A=5\) alone does not identify which means differ. The B factor has two levels, so its main-effect null concerns equality of the two B marginal means. Neither F statistic should be interpreted without its corresponding degrees of freedom and upper-tail reference distribution.
A Practical Audit Sequence
When reading a balanced two-factor table, apply the checks in order. This sequence separates arithmetic mistakes from interpretation mistakes.
Identify the numbers of levels \(a\) and \(b\), and the common cell size \(n\). If cell sizes are unequal, do not assume the balanced-table formulas apply unchanged.
Verify the four component formulas and confirm that they add to \(abn-1\).
Add the A, B, interaction, and error sums of squares and compare the result with the corrected total.
Divide each sum of squares by its own degrees of freedom. For a main-effect test, divide its mean square by \(MS_E\).
Use the correct numerator and denominator degrees of freedom, the upper-tail p-value, and the marginal-mean meaning of the main effect.
Common Mistakes to Avoid
- Confusing SS and MS. A sum of squares measures a total amount of variation; a mean square is that amount divided by its degrees of freedom. The F ratio uses mean squares.
- Using the wrong denominator. For the fixed-effects main-effect tests in this balanced model, compare \(MS_A\) or \(MS_B\) with \(MS_E\), not with another effect’s mean square.
- Ignoring the total row. The total is a useful audit: component sums of squares and component degrees of freedom should each add to their corresponding total.
- Overstating a main effect. A main-effect test is about marginal averages. It does not establish every level-by-level difference or make a causal claim by itself.
- Rounding too early. Keep full precision while computing mean squares, F statistics, and p-values; round the reported result only at the end. For example, the upper-tail p-value for \(F=12\) with \(df=(1,18)\) is approximately 0.002769.
Check Your Understanding
Answer each question by showing the check or interpretation that supports your conclusion.
- A balanced design has two A levels, four B levels, and three observations per cell. Find the degrees of freedom for A, B, the interaction, error, and corrected total. Do they add correctly?
- A table reports \(SS_A=30\), \(df_A=2\), \(SS_E=80\), and \(df_E=20\). Calculate \(MS_A\), \(MS_E\), and the A F statistic.
- Why is an ANOVA F-test p-value an upper-tail probability rather than a lower-tail probability?
- For a factor with four levels, what does a significant main-effect test establish, and what does it not establish about the four marginal means?
- A report says that a significant A main effect proves the A difference is the same at every B level. Explain why that statement goes beyond the main-effect test.