Why Test a Prespecified Contrast?
A one-way ANOVA can test whether all population group means are equal, but an omnibus result does not identify which particular pattern of means matters. A study may instead specify a focused question before collecting data: for example, whether the average response in two new treatment groups, taken together, exceeds the average response in two established groups. A prespecified contrast turns that question into one linear combination of population means, which can be tested directly.
This tutorial concerns a single planned contrast and its t test. The test uses the pooled within-group variation from the one-way ANOVA, but its hypotheses concern the chosen linear combination—not the omnibus hypothesis that every group mean is equal.
The zero-sum condition makes the combination a comparison of group means: adding the same constant to every \(\mu_i\) leaves \(\psi\) unchanged. For instance, \(c=(1,-1,0)\) compares the first two means, while \(c=(1,1,-1,-1)\) compares the sum of the first two means with the sum of the last two. The coefficient scale matters for the numerical value of \(\psi\), but multiplying every coefficient by the same nonzero constant also multiplies its estimate and standard error by the same absolute factor; the resulting t statistic is unchanged.
Model, Standard Error, and Test Statistic
Suppose observations in group \(i\) are independent and normally distributed with population mean \(\mu_i\) and common variance \(\sigma^2\). Observations from different groups are also independent. Let \(n_i\) be the sample size in group \(i\), \(N=\sum_i n_i\), and \(k\) the number of groups. The within-group sum of squares is \(SSE=\sum_i\sum_j(X_{ij}-\bar X_i)^2\), and the pooled residual mean square is \(MSE=SSE/(N-k)\). The residual degrees of freedom are \(\nu=N-k\).
Under this model, the variance of the estimated contrast is estimated by \(MSE\sum_i c_i^2/n_i\). Thus its estimated standard error and test statistic for \(H_0:\psi=\psi_0\) are:
For a contrast test, the usual null value is \(\psi_0=0\), but the formula also applies to a specified nonzero value. Unequal group sizes are allowed: each term in the standard error uses its own \(n_i\). The test is exact under the stated normal, independent, common-variance model. Without the common-variance or normality assumptions, this pooled-error t test is not generally an exact t test.
Why the Statistic Has a t Distribution
The numerator \(\hat{\psi}-\psi_0\), under the null, is a linear combination of independent normal sample means. It is therefore normal with mean zero and variance \(\sigma^2\sum_i c_i^2/n_i\). Meanwhile, the pooled within-group quantity \(SSE/\sigma^2\) has a chi-square distribution with \(N-k\) degrees of freedom, and it is independent of the group means. Substituting \(MSE=SSE/(N-k)\) into the denominator standardizes the normal numerator using this independent estimate of \(\sigma^2\). That is the defining construction of a Student t statistic.
More explicitly, if \(Z=(\hat{\psi}-\psi_0)/\{\sigma\sqrt{\sum_i c_i^2/n_i}\}\) and \(U=SSE/\sigma^2\), then under the null \(Z\) is standard normal, \(U\) is chi-square with \(\nu\) degrees of freedom, and they are independent. Therefore:
This justification also explains why the residual degrees of freedom, rather than the number of groups or the total sample size alone, determine the reference distribution.
Choosing the Alternative and the p-Value
The direction of the alternative must match the research question and be specified before looking at the results. Let \(T\) have a Student t distribution with \(\nu\) degrees of freedom. For an observed statistic \(t_{\mathrm{obs}}\), the p-value is the probability, assuming the null, of a statistic at least as extreme in the direction described by the alternative.
| Question | Hypotheses | p-value |
|---|---|---|
| Is the contrast different from the null value? | \(H_0:\psi=\psi_0\); \(H_A:\psi\ne\psi_0\) | \(2P(T\geq |t_{\mathrm{obs}}|)\) |
| Is the contrast greater than the null value? | \(H_0:\psi\leq\psi_0\); \(H_A:\psi>\psi_0\) | \(P(T\geq t_{\mathrm{obs}})\) |
| Is the contrast less than the null value? | \(H_0:\psi\geq\psi_0\); \(H_A:\psi<\psi_0\) | \(P(T\leq t_{\mathrm{obs}})\) |
The equality value is used to calculate the test statistic in a one-sided test; the composite null includes values on the other side as well. If the statistic points opposite to the stated direction, the one-sided p-value will be large rather than becoming small because its sign is ignored. A p-value is not the probability that the null hypothesis is true. At a chosen significance level \(\alpha\), reject the null when the p-value is at most \(\alpha\).
Worked Example: A Two-Sided Difference Between Two Groups
A hypothetical laboratory compares average completion times for three independent procedures. The sample sizes are \(n_1=2\), \(n_2=2\), and \(n_3=3\), and the sample means are \(14\), \(10\), and \(12\) minutes. The ANOVA within-group sum of squares is \(SSE=16\). Test whether the first two population means differ, using a two-sided test at \(\alpha=0.05\). Assume independent normal observations with a common variance, and suppose this contrast was prespecified.
Method. Use \(c=(1,-1,0)\), so \(\psi=\mu_1-\mu_2\) and the null value is \(\psi_0=0\). The coefficients sum to \(1-1+0=0\), as required.
Degrees of freedom and pooled error. There are \(N=2+2+3=7\) observations in \(k=3\) groups, so \(\nu=N-k=7-3=4\). Therefore \(MSE=16/4=4\).
Estimate and standard error. The estimate is \(\hat{\psi}=1(14)-1(10)+0(12)=4\) minutes. The coefficient-weighted sum is \(1^2/2+(-1)^2/2+0^2/3=1\), so \(SE(\hat{\psi})=\sqrt{4(1)}=2\) minutes.
Statistic, p-value, and decision. The statistic is \(t=(4-0)/2=2\), with \(4\) degrees of freedom. The two-sided p-value is \(2P(T_4\geq2)\approx0.1161\). Since \(0.1161>0.05\), do not reject the null; the data do not provide sufficient evidence of a difference at the 5% level.
Check and trap avoided. The numerator is the difference of the two means selected by the contrast, not the largest minus the smallest of all three means. The third group still contributes to the pooled residual degrees of freedom and error estimate, but its contrast coefficient is zero. The calculation consistently gives \(t=2\), \(4\) residual degrees of freedom, and a two-sided p-value greater than \(0.05\).
Worked Example: A One-Sided Comparison of Two Pairs
A hypothetical reliability study compares four designs, with two independent test units per design. The sample means are \(9\), \(11\), \(6\), and \(8\) hours of operation. The within-group sum of squares is \(SSE=8\). Before data collection, investigators specify that the first pair's combined mean should exceed the second pair's combined mean. Test that directional claim at \(\alpha=0.05\), assuming independent normal observations and a common variance.
Method. The specified contrast has coefficients \(c=(1,1,-1,-1)\), which sum to zero. The contrast is \(\psi=\mu_1+\mu_2-\mu_3-\mu_4\), and the hypotheses are \(H_0:\psi\leq0\) versus \(H_A:\psi>0\).
Error estimate and degrees of freedom. Here \(N=4(2)=8\) and \(k=4\), so \(\nu=8-4=4\). Then \(MSE=8/4=2\).
Estimate and standard error. The estimated contrast is \(9+11-6-8=6\) hours. The weighted sum is \(1/2+1/2+1/2+1/2=2\), so \(SE(\hat{\psi})=\sqrt{2(2)}=2\) hours.
Statistic, p-value, and decision. The statistic is \(t=(6-0)/2=3\), with \(4\) degrees of freedom. Because the alternative is greater than zero, use the upper tail only: \(p=P(T_4\geq3)\approx0.0201\). Since \(0.0201<0.05\), reject the null in favor of the specified greater-than alternative.
Check and trap avoided. The two pair means are compared as sums because the specified coefficients are \(1,1,-1,-1\); replacing them with coefficients \(1/2,1/2,-1/2,-1/2\) would rescale the contrast but leave \(t\) unchanged if applied consistently. The main trap is doubling this upper-tail probability: the prespecified alternative is one-sided, so the p-value is approximately \(0.0201\), not \(0.0401\).
Worked Example: Unequal Group Sizes and a Nonzero Contrast Estimate
A hypothetical study has three independent groups with sample sizes \(n_1=2\), \(n_2=3\), and \(n_3=2\), and sample means \(12\), \(8\), and \(10\). The pooled within-group sum of squares is \(SSE=24\). Investigators prespecify the contrast \(2\mu_1-\mu_2-\mu_3\) and test whether it differs from zero, using \(\alpha=0.05\). Assume normal observations with a common variance.
Method. The coefficients are \(c=(2,-1,-1)\) and sum to zero. This is a two-sided test of \(H_0:\psi=0\) against \(H_A:\psi\ne0\).
Degrees of freedom and error estimate. The total sample size is \(N=2+3+2=7\), so \(\nu=7-3=4\). Thus \(MSE=24/4=6\).
Estimate and standard error. The estimated contrast is \(2(12)-8-10=6\). The coefficient-weighted sum is \(2^2/2+(-1)^2/3+(-1)^2/2=2+1/3+1/2=17/6\). Consequently, \(SE(\hat{\psi})=\sqrt{6(17/6)}=\sqrt{17}\approx4.1231\).
Statistic, p-value, and decision. The statistic is \(t=6/\sqrt{17}\approx1.4552\), with \(4\) degrees of freedom. The two-sided p-value is \(2P(T_4\geq1.4552)\approx0.2193\). Since \(0.2193>0.05\), do not reject the null.
Check and trap avoided. The second group contributes \(1/3\) to the weighted sum because it has three observations, whereas each other group contributes \(1/2\). Using a common \(n\) in the standard error would be wrong for these unequal sample sizes. The computed standard error is approximately \(4.1231\), giving a statistic of approximately \(1.4552\), not a value obtained by treating all sample sizes as two.
Common Mistakes and Interpretation
A contrast test focuses on one planned question. It does not replace the ANOVA model assumptions, and it does not answer every possible question about the group means. Interpret the result in the direction and units of the contrast that was actually specified.
- Using the wrong degrees of freedom. For the pooled one-way ANOVA error estimate, use \(N-k\), not \(N-1\) or \(k-1\).
- Ignoring unequal sample sizes. The estimated variance includes \(c_i^2/n_i\) separately for each group.
- Using the wrong tail. A two-sided test uses both tails; a directional test uses the tail specified by its alternative. Do not decide the direction after seeing the sign of the observed contrast.
- Changing the contrast after seeing the means. The nominal p-value is for the contrast and testing procedure that were specified. Choosing a favorable contrast after examining results is not the same prespecified test.
- Confusing the contrast test with the omnibus ANOVA test. The contrast null concerns one linear combination. The omnibus null concerns equality of all group means; the two hypotheses need not lead to the same conclusion.
Check Your Understanding
For each problem, assume independent normal observations with a common variance unless stated otherwise. Use the residual mean square from the stated one-way ANOVA model.
- Three groups have sample sizes \(2,2,3\), means \(11,9,10\), and \(MSE=1\). For the prespecified contrast \(\mu_1-\mu_2\), test against a two-sided alternative. Find the estimate, standard error, t statistic, degrees of freedom, and p-value. (Answer: estimate \(2\), \(SE=1\), \(t=2\), \(df=4\), two-sided \(p\approx0.1161\).)
- Four groups each have sample size \(2\), means \(7,9,6,8\), and \(MSE=2\). For the contrast \(\mu_1+\mu_2-\mu_3-\mu_4\), find the t statistic and two-sided p-value. (Answer: estimate \(2\), \(SE=2\), \(t=1\), \(df=4\), \(p\approx0.3739\).)
- Three groups have sample sizes \(2,2,3\), means \(14,10,12\), and \(MSE=4\). For the prespecified contrast \(\mu_1-\mu_2\), test \(H_A:\mu_1-\mu_2>0\). Give the statistic, degrees of freedom, and one-sided p-value. (Answer: \(t=2\), \(df=4\), upper-tail \(p\approx0.0581\).)
- Three groups have sample sizes \(2,3,2\), means \(11,8,9\), and \(MSE=6\). For \(2\mu_1-\mu_2-\mu_3\), find the standard error, t statistic, and two-sided p-value. (Answer: estimate \(5\), \(SE=\sqrt{17}\approx4.1231\), \(t\approx1.2127\), \(df=4\), \(p\approx0.2919\).)
- Two groups have sample sizes \(3\) and \(3\), means \(12\) and \(9\), and \(MSE=2\). Test the prespecified contrast \(\mu_1-\mu_2\) against the nonzero null value \(\psi_0=1\), using a two-sided alternative. (Answer: \(\hat{\psi}=3\), \(SE=\sqrt{4/3}\approx1.1547\), \(t\approx1.7321\), \(df=4\), \(p\approx0.1583\).)