Why Sums of Squares Invite Misinterpretation
An ANOVA table can make the analysis look like a simple accounting exercise: each model term gets a sum of squares, and those entries appear to divide the response’s variation among the terms. In an unbalanced factorial design, that reading can be misleading. Different sums-of-squares methods can allocate overlapping information differently, and a label such as “Type III” does not, by itself, tell you what scientific comparison was tested.
The useful question is not just “How large is this sum of squares?” It is also “Compared with which model, and for which hypothesis?” These questions keep a numerical output from being mistaken for a unique measure of a term’s importance or for an answer to a different scientific question.
For nested models fitted to the same response observations, the extra sum of squares for adding terms is the reduction in residual sum of squares:
This reduction is conditional on the terms already in the reduced model. It is not automatically a unique share of the total variation. Sequential (Type I) sums of squares use this idea by adding terms in a specified order. Partial tests compare a full model with a reduced model that omits the term or group of terms being tested. Different comparisons can therefore yield different sums of squares without any arithmetic error.
When a Sum of Squares Is Not a Unique Contribution
If model terms are correlated in the design, they can explain some of the same variation in the response. Assigning that shared information to one term or another depends on the comparison being made. A sequential sum of squares asks what a term adds at its position in the chosen order. Reversing the order can change the answer. This is a feature of the comparisons, not a defect in the arithmetic.
Worked Example: The Order Changes a Sequential Allocation
Consider three hypothetical observations and two centered model-term vectors. Let \(u=(1,-1,0)/\sqrt{2}\) and \(v=(1,1,-2)/\sqrt{6}\), and suppose the response vector is \(y=u\). The two terms are represented by \(x=u\) and \(z=(u+v)/\sqrt{2}\). Both term vectors sum to zero, so they are orthogonal to the intercept. Also, \(u\) and \(v\) have length 1 and are orthogonal.
The total sum of squares about the mean is \(y'y=1\). If \(x\) enters first, its projection explains all the response variation:
There is then no residual variation for \(z\) to explain, so the sequential sum of squares for \(z\), added second, is \(0\). If \(z\) enters first instead, its sequential sum of squares is:
Adding \(x\) after \(z\) explains the remaining \(1/2\), because the model containing both spans the directions \(u\) and \(v\), including \(y=u\). Thus the two term orders allocate the total \(1\) differently: \(1,0\) in one order and \(1/2,1/2\) in the other. This small projection example has no residual degrees of freedom and is not an inferential test; it demonstrates why sequential sums of squares are order-dependent.
In factorial ANOVA, the same issue arises when terms share information because of unequal cell sizes or because a model includes related effects. A common mistake is to call a term’s reported sum of squares “the variation uniquely explained by that term.” The correction is to name the comparison: for example, “the reduction in residual sum of squares when this term is added after these specified terms.” Unless the terms are orthogonal under the design, the table entry need not be an exclusive allocation.
Why Type III Is Not Universally Preferable
Type III sums of squares are often used to test a term after accounting for the other terms in a model. In a complete factorial design with an interaction, a Type III main-effect test commonly corresponds to an equal-weight comparison across the levels of the other factor. That can be a useful target, but it is not automatically the target that matters. In an unbalanced design, an equal-weight marginal comparison can differ from a comparison weighted by the observed cell sizes. Neither weighting is universally correct: the scientific question determines which is relevant.
Worked Example: The Weighting Rule Changes the Main-Effect Comparison
Suppose a hypothetical two-factor study has response means and sample sizes as shown. Factor A has levels A1 and A2; factor B has levels B1 and B2. The model includes the A-by-B interaction, and all four cells are observed.
| Cell | Sample size | Response mean |
|---|---|---|
| A1, B1 | 2 | 10 |
| A1, B2 | 8 | 14 |
| A2, B1 | 8 | 13 |
| A2, B2 | 2 | 15 |
First consider the observed, sample-size-weighted mean for each A level. For A1, the total response across its cells is \(2(10)+8(14)=132\), from 10 observations, giving \(132/10=13.2\). For A2, the total is \(8(13)+2(15)=134\), also from 10 observations, giving \(134/10=13.4\). The difference A2 minus A1 is \(13.4-13.2=0.2\) response units.
Now compare the A levels by averaging equally over B. A1’s equal-weight mean is \((10+14)/2=12\), and A2’s is \((13+15)/2=14\). The equal-weight difference is \(14-12=2\) response units. These are different comparisons, although both use the same four cell means.
The difference occurs because the observed cell sizes give more weight to B2 for A1 and to B1 for A2, while equal weighting gives each B level the same weight within each A level. A Type III test might address the equal-weight marginal comparison in this complete factorial model, depending on the specified coding and hypothesis. It is not therefore a universally better answer than a differently weighted comparison. To choose, state the target comparison first and then verify that the test corresponds to it.
A related error is to assume that a method’s name makes its result intrinsically more valid. Type I, Type II, and Type III tests answer different model-comparison questions in settings where those questions differ. The choice is not a contest in which one type always wins; it depends on the model and the hypothesis of interest. In particular, the presence of an interaction can make a single overall main-effect summary insufficient to describe how the factor behaves across the other factor’s levels.
A Software Label Is Not the Hypothesis
A software table may display a Type III sum of squares, but the label is not a substitute for reading the actual hypothesis. Coding conventions, model terms, and the hypothesis matrix determine what is tested. In a model with an interaction, a test of one coefficient can represent a simple effect at a reference level, while a test of a whole main-effect term can represent a different, marginal comparison. Check the tested linear combination of parameters—or the equivalent comparison of cell means—before interpreting the p-value.
Worked Example: A Coefficient Test and a Marginal Test Differ
Use the four hypothetical cell means from the previous example: \(\mu_{11}=10\), \(\mu_{12}=14\), \(\mu_{21}=13\), and \(\mu_{22}=15\), where the first index denotes A and the second denotes B. Under treatment coding with A1 and B1 as reference levels, the coefficient for A represents the difference between A2 and A1 specifically at B1.
That simple effect is \(\mu_{21}-\mu_{11}=13-10=3\) response units. The simple effect at B2 is instead \(\mu_{22}-\mu_{12}=15-14=1\) response unit. The equal-weight marginal A difference is the average of these two simple effects:
Thus, a test of the treatment-coded A coefficient tests whether the A difference at B1 is zero, whereas a test of the equal-weight marginal A effect tests whether the average of the two A differences is zero. Those are not the same hypothesis. The cell means here show the distinction in the quantities being tested; they do not provide an error variance or p-value, so no significance conclusion follows from these differences alone.
When reviewing software output, do not infer the hypothesis from “Type III” alone. Check the factor coding, the model terms, and the contrast or linear hypothesis used for the test. Then state the result in terms of that comparison, in response units and in the context of the study.
A Practical Check Before Interpreting an ANOVA Table
A careful interpretation separates three questions that are easily conflated: what model was fitted, what reduced model or contrast defines the test, and what scientific comparison that represents. A small p-value addresses the specified null hypothesis under the model assumptions; it does not make the term’s sum of squares a unique share of variation, nor does it validate the choice of hypothesis.
Include interactions and note the factor coding. A test cannot be understood independently of the model in which it appears.
For a nested-model test, state which terms are removed from the full model. For a contrast test, state the exact linear combination or cell-mean comparison.
Describe it as a reduction in residual sum of squares for that comparison, not automatically as a unique contribution.
Check whether the test uses observed-size weighting, equal weighting, a simple effect, or another specified comparison, and report that target plainly.
A common pitfall is to write, “Factor A uniquely explains this much of the response variation because its Type III sum of squares is large.” This reasoning is wrong because the sum of squares is defined by a particular model and hypothesis; other terms may share explanatory information, and the test may target a contrast other than the one implied by the sentence. A more defensible report names the tested effect, the weighting or comparison if relevant, and the test result. If the scientific target is unclear, the first task is to clarify it—not to select a sums-of-squares type by reputation.
Check Your Understanding
Use the questions below to check whether you can distinguish a sums-of-squares label from the comparison it represents.
- In your own words, why can sequential sums of squares change when the order of model terms changes?
- What does the extra sum of squares \(SSE_{\text{reduced}}-SSE_{\text{full}}\) describe, and why is it not necessarily a unique contribution?
- In the worked example, why did the equal-weight and observed-size-weighted A comparisons differ?
- With an interaction in a treatment-coded model, what comparison does the A coefficient represent when A1 and B1 are the reference levels?
- Name two details you should check before interpreting a software row labelled “Type III.”