Why Compare Pairs of Means?
A one-way ANOVA evaluates whether the population group means are all equal, but its omnibus test does not identify which groups differ. Pairwise comparisons answer that more specific question: for each chosen pair, how large is the difference between their population means, and how much evidence is there against equality? Each such difference is a contrast, so the contrast framework gives a consistent way to estimate and test it.
Here the focus is the unadjusted t test for each pair using the pooled error estimate from a one-way ANOVA. “Unadjusted” means that each test is carried out at its stated level without changing its p-value or significance threshold to account for the number of pairs tested. Thus, these calculations do not by themselves provide familywise error control when many pairs are examined.
Model and Pairwise Contrasts
Suppose there are \(k\) independent groups. In group \(i\), the observations are \(Y_{i1},\ldots,Y_{in_i}\), with population mean \(\mu_i\). The one-way normal model assumes that observations are independent and that \(Y_{ij}\sim N(\mu_i,\sigma^2)\): group means may differ, but all groups share the same variance \(\sigma^2\). The pooled-error t procedure also relies on this common-variance model. If observations are dependent or group variances are unequal, the standard pooled-error calculation may not be appropriate.
For example, with three groups, \(\mu_2-\mu_1\) has coefficient vector \((-1,1,0)\). The contrast is oriented: reversing the order gives \(\mu_1-\mu_2\), which has the opposite estimate and test statistic. Both comparisons concern the same equality null hypothesis, but the sign must match the stated direction.
Let \(N=\sum_{i=1}^k n_i\), and let \(SSE\) be the sum of squared deviations of observations from their own group means. The pooled error mean square is \(MSE=SSE/(N-k)\). Under the common-variance model, \(MSE\) estimates \(\sigma^2\), and its degrees of freedom are the residual degrees of freedom from the one-way ANOVA.
To test \(H_0:\mu_i-\mu_j=0\), calculate the observed difference divided by its pooled-error standard error. Under the null and the model assumptions, this statistic has a t distribution with \(N-k\) degrees of freedom. A two-sided test uses both tails; a one-sided test uses the tail determined by the alternative hypothesis.
Why the Standard Error Has This Form
The pairwise standard error follows from the variance of a difference of independent sample means. Under the model, each observation has variance \(\sigma^2\), so the variance of the mean of \(n_i\) observations in group \(i\) is \(\sigma^2/n_i\). Independence between groups means the covariance of the two sample means is zero. Subtracting the means therefore adds their variances.
\(\operatorname{Var}(\bar{Y}_i)=\sigma^2/n_i\) and \(\operatorname{Var}(\bar{Y}_j)=\sigma^2/n_j\).
\(\operatorname{Var}(\bar{Y}_i-\bar{Y}_j)=\operatorname{Var}(\bar{Y}_i)+\operatorname{Var}(\bar{Y}_j)=\sigma^2(1/n_i+1/n_j)\).
Replace \(\sigma^2\) by the pooled estimate \(MSE\), then take the square root to obtain the standard error.
This also explains why the sample sizes for the two groups matter separately. A comparison involving a smaller group generally has a larger standard error, even when the estimated common variance is unchanged. The other groups contribute to the pooled estimate \(MSE\), but their sample sizes do not enter the pair’s standard-error formula.
Worked Pairwise Comparisons
Worked Example: Testing Every Pair in a Three-Group Study
A hypothetical study compares mean response times for three independent training methods. The observed sample means are \(\bar{Y}_A=10\), \(\bar{Y}_B=13\), and \(\bar{Y}_C=11\), in seconds, with \(n_A=n_B=n_C=4\). The within-group sum of squares is \(SSE=18\). Assume independent normal observations with a common variance. Calculate the estimate, standard error, and unadjusted two-sided t test for every pair at the 5% level.
Method and pooled error. There are \(N=4+4+4=12\) observations and \(k=3\) groups, so the error degrees of freedom are \(12-3=9\). Thus \(MSE=18/9=2\).
Groups A and B. The contrast is \(\mu_A-\mu_B\), with coefficients \((1,-1,0)\). Its estimate is \(10-13=-3\) seconds. Its standard error is \(\sqrt{2(1/4+1/4)}=\sqrt{1}=1\). Therefore \(t=-3/1=-3.000\), with 9 degrees of freedom; the two-sided p-value is approximately \(0.015\). This pair is significant at the unadjusted 5% level.
Groups A and C. The contrast is \(\mu_A-\mu_C\), with coefficients \((1,0,-1)\). Its estimate is \(10-11=-1\) second. Its standard error is again \(\sqrt{2(1/4+1/4)}=1\), giving \(t=-1/1=-1.000\). With 9 degrees of freedom, the two-sided p-value is approximately \(0.343\), so this pair is not significant at the unadjusted 5% level.
Groups B and C. The contrast is \(\mu_B-\mu_C\), with coefficients \((0,1,-1)\). Its estimate is \(13-11=2\) seconds. Its standard error is \(1\), so \(t=2/1=2.000\). With 9 degrees of freedom, the two-sided p-value is approximately \(0.077\), so this pair is not significant at the unadjusted 5% level.
Check. Reversing any pair’s order would change the sign of its estimate and t statistic but not its two-sided p-value. The calculations use the same \(MSE=2\) and residual degrees of freedom for all three pairs.
Worked Example: Unequal Group Sizes Change the Standard Errors
A hypothetical comparison of three materials gives sample sizes \(n_A=4\), \(n_B=6\), and \(n_C=8\), and sample means \(\bar{Y}_A=20\), \(\bar{Y}_B=23\), and \(\bar{Y}_C=26\). The within-group sum of squares is \(SSE=54\). Assume the independent, normal, common-variance model. Find the pooled-error estimates and unadjusted two-sided tests for all three pairs.
Method and pooled error. Here \(N=4+6+8=18\), \(k=3\), and the error degrees of freedom are \(18-3=15\). Therefore \(MSE=54/15=3.6\).
A versus B. For \(\mu_B-\mu_A\), the estimate is \(23-20=3\). The standard error is \(\sqrt{3.6(1/6+1/4)}=\sqrt{3.6(5/12)}=\sqrt{1.5}\approx1.225\). Thus \(t=3/1.225\approx2.449\), with 15 degrees of freedom. The two-sided p-value is approximately \(0.027\), so this comparison is significant at the unadjusted 5% level.
A versus C. For \(\mu_C-\mu_A\), the estimate is \(26-20=6\). The standard error is \(\sqrt{3.6(1/8+1/4)}=\sqrt{3.6(3/8)}=\sqrt{1.35}\approx1.162\). Thus \(t=6/1.162\approx5.164\), with 15 degrees of freedom, and the two-sided p-value is less than \(0.001\).
B versus C. For \(\mu_C-\mu_B\), the estimate is \(26-23=3\). The standard error is \(\sqrt{3.6(1/8+1/6)}=\sqrt{3.6(7/24)}=\sqrt{1.05}\approx1.025\). Thus \(t=3/1.025\approx2.927\), with 15 degrees of freedom, and the two-sided p-value is approximately \(0.010\).
Check. Each estimate equals the corresponding difference of sample means, and each standard error uses the two sample sizes in that pair. The standard errors are not all equal despite the common pooled variance, because the group sizes differ.
Worked Example: Matching the Test Direction to the Contrast
A hypothetical four-group assessment has group sizes \(6,8,8,10\), sample means \(18,20,19,21\), and within-group sum of squares \(SSE=112\), in the same order \(A,B,C,D\). Assume independent normal observations with a common variance. Test whether group D has a greater population mean than group A, using an unadjusted one-sided test at level \(0.05\).
Method and pooled error. The total sample size is \(N=6+8+8+10=32\), and \(k=4\), so the error degrees of freedom are \(32-4=28\). Then \(MSE=112/28=4\).
The stated alternative is \(\mu_D-\mu_A>0\), so use the contrast with coefficients \((-1,0,0,1)\). Its estimate is \(\bar{Y}_D-\bar{Y}_A=21-18=3\). Its standard error is \(\sqrt{4(1/10+1/6)}=\sqrt{4(8/30)}=\sqrt{16/15}\approx1.033\).
The test statistic is \(t=3/1.033\approx2.905\), with 28 degrees of freedom. The upper-tail p-value is approximately \(0.0035\), so the test rejects \(H_0:\mu_D-\mu_A=0\) in favor of the stated alternative at the unadjusted 5% level.
Check. If the contrast had instead been written as \(\mu_A-\mu_D\), its estimate and t statistic would be \(-3\) and approximately \(-2.905\). The lower-tail test would give the same evidence. Using that reversed contrast with an upper-tail alternative would be a direction error.
Reading Results and Avoiding Common Errors
A pairwise t statistic measures the estimated difference in standard-error units. Its sign describes the direction of the ordered contrast; its magnitude, together with the residual degrees of freedom, determines the p-value. For a two-sided test, the p-value is the probability under the null model of a t statistic at least as far from zero in either direction. For a one-sided test, the tail must match the alternative.
The following errors are especially easy to make when calculating several pairs:
- Using the wrong contrast order. If the question asks whether group B exceeds group A, define the estimate as \(\bar{Y}_B-\bar{Y}_A\), not the reverse.
- Using the wrong degrees of freedom. The pooled-error test uses \(N-k\), not \(N-1\), because the ANOVA estimates a separate mean for each of the \(k\) groups.
- Using a single group variance instead of pooled error. The stated procedure uses \(MSE\) from all within-group variation, not a variance calculated from only the two groups being compared.
- Forgetting unequal sample sizes. The standard error contains both \(1/n_i\) and \(1/n_j\); the balanced-group shortcut cannot be used when the sizes differ.
- Treating unadjusted results as simultaneous conclusions. Each p-value answers its own test. Examining many pairs without an adjustment does not guarantee that the chance of at least one false rejection stays at the nominal level.
For example, with unequal group sizes, replacing \(1/n_i+1/n_j\) by \(2/n_i\) is generally wrong. That shortcut is valid only when the two compared groups have the same size \(n_i=n_j\). It can yield an incorrect standard error, test statistic, and conclusion.
Check Your Understanding
For each problem, assume independent normal observations with a common variance and use the pooled-error t procedure specified.
- Three groups have sizes \(4,6,5\), sample means \(\bar{Y}_A=7\) and \(\bar{Y}_B=10\), and pooled \(MSE=5\). For the contrast \(\mu_B-\mu_A\), calculate the estimate, standard error, t statistic, and degrees of freedom. (Answer: estimate \(3\); standard error \(\sqrt{25/12}\approx1.443\); \(t\approx2.078\); \(df=12\).)
- Groups A, B, and C have sizes \(8,12,10\), means \(40,42,43\), and \(SSE=90\). For \(\mu_B-\mu_A\), calculate the estimate, standard error, and t statistic. (Answer: \(N=30\), \(df=27\), \(MSE=90/27=10/3\); estimate \(2\); \(SE=\sqrt{(10/3)(1/8+1/12)}=\sqrt{25/72}\approx0.589\); \(t\approx3.394\).)
- Three groups have sizes \(5,7,8\), means \(12.4,15.4,14.1\), and \(MSE=2.5\). For \(\mu_B-\mu_A\), calculate the standard error, t statistic, and two-sided p-value using \(df=17\). (Answer: estimate \(3\); \(SE=\sqrt{2.5(1/7+1/5)}=\sqrt{6/7}\approx0.926\); \(t\approx3.240\); two-sided \(p\approx0.005\).)
- Three groups have four observations each, means \(5,7,8\), and \(MSE=1.5\). At the unadjusted two-sided 5% level, which pairwise comparisons reject equality? Use \(df=9\). (Answer: all pair standard errors are \(\sqrt{1.5(1/4+1/4)}\approx0.866\); absolute t statistics are approximately \(2.309,3.464,1.155\) for A–B, A–C, B–C. A–B and A–C reject; B–C does not.)
- For a one-sided test of whether group D has a higher mean than group A, the observed means are \(\bar{Y}_D=31\) and \(\bar{Y}_A=28\), with standard error \(1.2\). State the ordered contrast and calculate the test statistic. (Answer: contrast \(\mu_D-\mu_A\); estimate \(3\); \(t=3/1.2=2.5\), using the upper tail.)