Tukey-Kramer Comparisons for Unequal Group Sizes

Why Unequal Group Sizes Need a Pairwise Adjustment

A one-way ANOVA compares the means of several groups, but a significant omnibus F test does not identify which groups differ. When the groups have different sample sizes, every pair of sample means has a different amount of sampling variability. Tukey-Kramer comparisons account for this while controlling the chance of making one or more false rejections across the full set of pairwise comparisons.

The procedure uses the pooled error estimate from the one-way ANOVA and a critical value from the studentized range distribution. The calculation is similar to Tukey HSD for balanced designs, but it adjusts the comparison standardizer for each pair's two sample sizes.

Definition: Suppose there are $k$ independent groups, with sample means $\bar{x}_1,\ldots,\bar{x}_k$ and sample sizes $n_1,\ldots,n_k$. Let $MSE$ be the one-way ANOVA error mean square and $df_{\text{error}}$ its degrees of freedom. For groups $i$ and $j$, the Tukey-Kramer statistic is $$ q_{ij}=\frac{|\bar{x}_i-\bar{x}_j|}{\sqrt{\frac{MSE}{2}\left(\frac{1}{n_i}+\frac{1}{n_j}\right)}}. $$ At familywise significance level $\alpha$, compare this with the studentized-range critical value $q_{\alpha;k,df_{\text{error}}}$. Reject the equality of the two group means when $q_{ij}>q_{\alpha;k,df_{\text{error}}}$.

There are $k(k-1)/2$ distinct pairs. The same critical value is used for all of them because it is based on the total number of groups $k$, the ANOVA error degrees of freedom, and the chosen familywise level $\alpha$. The pair-specific part is the denominator, which changes with $n_i$ and $n_j$.

Model and Pooled Error Estimate

Tukey-Kramer comparisons use the one-way ANOVA model: each observation in group $i$ is its group mean plus an error term. The errors are assumed to be independent, normally distributed with mean zero, and to have the same variance $\sigma^2$ in every group. Independence should be justified by the study design; approximate normality and similar within-group variability should be reasonable for the data. These are the standard conditions for the one-way ANOVA model.

The ANOVA error mean square estimates the common error variance by pooling the within-group variation. If group $i$ has sample size $n_i$, then $N=\sum_i n_i$ is the total sample size, and the error degrees of freedom are $N-k$. In terms of the within-group sum of squares $SSE$,

MSE=\frac{SSE}{N-k}, \qquad df_{\text{error}}=N-k.

Use this pooled $MSE$, not a separate variance estimate for each pair. The procedure relies on the equal-variance model to treat the group errors as having one common variance.

Why the Pair-Specific Denominator Has This Form

Under the model, the variance of the sample mean in group $i$ is $\sigma^2/n_i$. Since two different groups are independent, the variance of their difference is the sum of those variances:

\operatorname{Var}(\bar{X}_i-\bar{X}_j) =\operatorname{Var}(\bar{X}_i)+\operatorname{Var}(\bar{X}_j) =\sigma^2\left(\frac{1}{n_i}+\frac{1}{n_j}\right).

Replacing $\sigma^2$ with its pooled estimate $MSE$ gives the estimated standard error of the difference, $\sqrt{MSE(1/n_i+1/n_j)}$. The studentized-range statistic uses this standard error divided by $\sqrt{2}$, which gives the denominator in the Tukey-Kramer statistic. For equal sample sizes $n_i=n_j=n$, that denominator reduces to $\sqrt{MSE/n}$, the Tukey HSD standardizer. With unequal sizes, each pair instead gets its own denominator.

Worked Example: Comparing All Pairs in Three Unequal Groups

A hypothetical experiment compares mean response across three treatments. The sample sizes are $n_1=10$, $n_2=15$, and $n_3=18$; the sample means are 5.2, 7.0, and 9.0 response units. The ANOVA error mean square is $MSE=2.00$, with $df_{\text{error}}=40$. Suppose the studentized-range table or software gives $q_{0.05;3,40}=3.44$. The independent design and residual checks are stipulated to support the model assumptions.

For groups 1 and 2, $$ \sqrt{\frac{2.00}{2}\left(\frac{1}{10}+\frac{1}{15}\right)} =\sqrt{0.166667} \approx 0.4082, \qquad q_{12}=\frac{|5.2-7.0|}{\sqrt{0.166667}} =\frac{1.8}{\sqrt{0.166667}} \approx 4.409. $$ Since $4.409>3.44$, reject equality of the treatment 1 and treatment 2 means.

For groups 1 and 3, $$ \sqrt{\frac{2.00}{2}\left(\frac{1}{10}+\frac{1}{18}\right)} =\sqrt{0.155556} \approx 0.3944, \qquad q_{13}=\frac{3.8}{\sqrt{0.155556}} \approx 9.635. $$ Since $9.635>3.44$, reject equality of the treatment 1 and treatment 3 means.

For groups 2 and 3, $$ \sqrt{\frac{2.00}{2}\left(\frac{1}{15}+\frac{1}{18}\right)} =\sqrt{0.122222} \approx 0.3496, \qquad q_{23}=\frac{2.0}{\sqrt{0.122222}} \approx 5.721. $$ Since $5.721>3.44$, reject equality of the treatment 2 and treatment 3 means as well. Thus all three pairwise comparisons are significant at the familywise 5% level under the stated model. The computations use unrounded radicands in the divisions; the displayed rounded values are for readability.

Carrying Out the Procedure

A reliable calculation separates the ANOVA quantities from the pairwise calculations. First establish the common error estimate and its degrees of freedom; then calculate a statistic for each distinct pair and compare each statistic with the same critical value.

Check the model.
Consider independence from the design, and whether the within-group distributions and variability are reasonably compatible with the one-way ANOVA assumptions.

Find the pooled error quantities.
Use $MSE=SSE/(N-k)$ and $df_{\text{error}}=N-k$, or obtain them from the ANOVA output.

Obtain the critical value.
For the specified familywise level $\alpha$, find $q_{\alpha;k,df_{\text{error}}}$ using the total number of groups and the error degrees of freedom.

Calculate each pair statistic.
For each of the $k(k-1)/2$ pairs, use that pair's sample sizes in the Tukey-Kramer denominator.

Make and report the decisions.
Reject the equality of a pair's means when its statistic exceeds the common critical value. Name the groups and state the familywise significance level.

Worked Example: A Pair That Does Not Pass the Tukey-Kramer Cutoff

In a hypothetical four-group study, two of the groups have sample sizes $n_1=6$ and $n_2=10$, with sample means 12 and 15 units. The ANOVA output reports $MSE=4.50$ and $df_{\text{error}}=24$. Suppose the 5% studentized-range critical value for $k=4$ and 24 error degrees of freedom is 3.90. Assume the study design supports independence and the residual behavior is compatible with the model.

For this pair, the Tukey-Kramer denominator is $$ \sqrt{\frac{4.50}{2}\left(\frac{1}{6}+\frac{1}{10}\right)} =\sqrt{2.25(0.266667)} =\sqrt{0.600} \approx 0.7746. $$ The statistic is $$ q_{12}=\frac{|12-15|}{\sqrt{0.600}} =\frac{3}{\sqrt{0.600}} \approx 3.873. $$ Because $3.873<3.90$, do not reject equality of these two group means in the Tukey-Kramer family of comparisons at the 5% level. This decision does not establish that the means are identical; it says that this comparison does not exceed the procedure's familywise-adjusted threshold.

The ordinary estimated standard error of this difference is $\sqrt{4.50(1/6+1/10)}=\sqrt{1.200}\approx1.095$. It is not the denominator to compare directly with a studentized-range critical value. The Tukey-Kramer denominator is that ordinary standard error divided by $\sqrt{2}$, or $\sqrt{0.600}$ here.

Finding the ANOVA Quantities Before Comparing Means

When the ANOVA output gives $SSE$ rather than $MSE$, calculate $MSE$ before doing any pairwise work. The error degrees of freedom depend on the total number of observations and the number of groups, not on the size of the particular pair being compared. The following example shows why keeping those quantities separate matters.

Worked Example: From Error Sum of Squares to Pairwise Decisions

A hypothetical one-way ANOVA has three groups with sample sizes 8, 12, and 10, so $N=30$. Its within-group error sum of squares is $SSE=72.0$. The group means are 18, 20, and 21 units. Suppose design information and residual checks support independence, approximate normality, and similar group variances. Use a 5% familywise level and the supplied critical value $q_{0.05;3,27}=3.51$.

First, the error degrees of freedom are $30-3=27$, and therefore $$ MSE=\frac{72.0}{27}=2.6667. $$ For groups 1 and 2, $$ q_{12}=\frac{2}{\sqrt{\frac{2.6667}{2}(1/8+1/12)}} =\frac{2}{\sqrt{1.33335(0.208333)}} \approx \frac{2}{0.5270} \approx 3.795. $$ This exceeds 3.51, so reject equality of the first two means.

For groups 1 and 3, the sample-mean difference is 3 units: $$ q_{13}=\frac{3}{\sqrt{1.33335(1/8+1/10)}} =\frac{3}{\sqrt{0.3000}} \approx 5.477. $$ This exceeds 3.51, so reject equality of the first and third means.

For groups 2 and 3, the difference is 1 unit: $$ q_{23}=\frac{1}{\sqrt{1.33335(1/12+1/10)}} =\frac{1}{\sqrt{0.24445}} \approx 2.023. $$ This is below 3.51, so do not reject equality of the second and third means. These decisions use the same $MSE$, error degrees of freedom, and critical value for all pairs; the pair-specific sample sizes determine each denominator.

What the Adjustment Does and Does Not Say

The studentized-range cutoff accounts for examining many pairwise differences, rather than treating each as an isolated test at level $\alpha$. Under the model assumptions, the Tukey-Kramer procedure controls the probability of making at least one false rejection among the full set of pairwise comparisons at approximately the chosen familywise level. It is intended for all pairwise mean comparisons. It is not a method for selecting only one or two comparisons chosen after seeing the results.

The decision is about evidence for a difference in population means, in the response's units and in the context of the groups. A rejected comparison does not establish that group membership caused the difference; causal claims require an appropriate study design. A non-rejected comparison does not prove equal means. It indicates only that the observed difference did not cross the familywise-adjusted threshold.

The procedure uses the pooled ANOVA error variation and depends on the common-variance assumption. If group variances differ substantially, or the design does not support independence, the standard Tukey-Kramer results may not be reliable. The sample sizes alone do not guarantee that the assumptions hold.

Common Mistakes

Using the wrong group count for the critical value. The critical value uses $k$, the number of all groups, not just the two groups in a particular comparison.
Using a pair's degrees of freedom. Use the ANOVA error degrees of freedom $N-k$ for every pair, not a separate degrees-of-freedom calculation for each pair.
Forgetting the factor of one-half. The Tukey-Kramer denominator is $\sqrt{\frac{MSE}{2}(1/n_i+1/n_j)}$. Using $\sqrt{MSE(1/n_i+1/n_j)}$ gives the ordinary standard error of the difference, not the studentized-range standardizer.
Reusing one denominator for every pair. With unequal group sizes, calculate a new denominator for each pair using those two groups' sample sizes.
Interpreting a non-rejection as proof of equality. The correct conclusion is that the comparison did not provide sufficient evidence of a difference at the familywise-adjusted level.

Key takeaway: For unequal group sizes, Tukey-Kramer uses the one-way ANOVA $MSE$ and $df_{\text{error}}=N-k$, together with a pair-specific denominator $\sqrt{\frac{MSE}{2}(1/n_i+1/n_j)}$. Compare every pair's statistic with the studentized-range critical value for the full set of $k$ groups.

Check Your Understanding

Use the definitions and procedure above to answer these questions.

For $k$ groups and total sample size $N$, what error degrees of freedom are used in Tukey-Kramer comparisons?
Why does each pair of groups have its own denominator when sample sizes are unequal?
A pair has $n_i=8$, $n_j=12$, and $MSE=3.0$. Write the Tukey-Kramer denominator for this pair. What additional quantity is needed to calculate its statistic?
Why is the ordinary standard error of a difference not itself the denominator in the studentized-range comparison?
A pair's statistic is below the critical value. State a correct conclusion and one conclusion that would be too strong.