A Worked Workshop in One-Way Planned Contrasts

From Research Questions to Contrast Inference

A one-way ANOVA study may include several treatment groups, but a research question often asks about one particular comparison among their population means. For example, researchers may want to know whether the average response across several active treatments differs from a control, or whether the response changes linearly across ordered dose levels. A planned contrast turns each question into a weighted combination of treatment means that can be estimated, tested, and reported directly.

This workshop follows those steps on one hypothetical dataset. The calculations use the pooled within-group variation from the one-way ANOVA model. Each test and interval below concerns one specified contrast; it is not a simultaneous procedure for a whole collection of comparisons.

Model and Contrast Calculations

Suppose there are \(a\) treatment groups, with \(n_i\) independent observations in group \(i\), for a total of \(N=\sum_{i=1}^a n_i\) observations. The one-way fixed-effects model is

Y_{ij}=\mu_i+\epsilon_{ij},\qquad \epsilon_{ij}\overset{\text{ind}}{\sim}N(0,\sigma^2), \quad i=1,\ldots,a,\quad j=1,\ldots,n_i.

Here \(\mu_i\) is the population mean response for treatment \(i\), and the errors are independent and normally distributed with a common variance. The response must be quantitative, and observations must be independent. In practice, random assignment or an appropriate sampling design supports independence; residual plots and subject-matter knowledge help assess the common-variance and normal-error assumptions. These assumptions matter especially when group sizes are small.

Definition: A contrast of the treatment means is \(L=\sum_{i=1}^a c_i\mu_i\), where the coefficients are fixed in advance and satisfy \(\sum_{i=1}^a c_i=0\). Its estimate is \(\hat{L}=\sum_{i=1}^a c_i\bar{x}_i\), where \(\bar{x}_i\) is the sample mean in group \(i\). The zero-sum condition makes the quantity a comparison of means rather than a common shift of all means.

Let \(SSE=\sum_i\sum_j(Y_{ij}-\bar{x}_i)^2\) be the within-group sum of squares. The pooled estimate of the common error variance is \(MSE=SSE/(N-a)\), with \(df=N-a\). Under the model, the standard error of a contrast estimate is

SE(\hat{L})=\sqrt{MSE\sum_{i=1}^a\frac{c_i^2}{n_i}}.

For one test of \(H_0:L=0\) against a two-sided alternative \(H_a:L\ne0\), the statistic is \(t=\hat{L}/SE(\hat{L})\), with \(N-a\) degrees of freedom under \(H_0\). A \(100(1-\alpha)\%\) confidence interval is \(\hat{L}\pm t_{1-\alpha/2,N-a}SE(\hat{L})\). The sign and units of the contrast estimate follow from the chosen coefficients and the units of the response.

One Hypothetical Dataset

Suppose a hypothetical experiment compares a control with three ordered treatment levels, called Level 1, Level 2, and Level 3. The quantitative response is measured in points, with five independent observations in each group. The observations are:

Group	Five hypothetical responses (points)	Sample mean (points)
Control	18, 19, 20, 21, 22	20
Level 1	21, 22, 23, 24, 25	23
Level 2	23, 24, 25, 26, 27	25
Level 3	26, 27, 28, 29, 30	28

For this dataset, \(a=4\), \(N=20\), and \(df=20-4=16\). In each group, deviations from its sample mean are \(-2,-1,0,1,2\), so the within-group sum of squares is \(4+1+0+1+4=10\) per group. Therefore

SSE=4(10)=40,\qquad MSE=\frac{40}{16}=2\text{ points}^2.

The equal residual sums in these groups do not establish that population variances are identical, but they are consistent with the common-variance model. The design stipulates independent observations. Normality is a modeling assumption here; five observations per group are too few to establish it convincingly from the data alone. We proceed with the stated one-way model and interpret the results subject to those assumptions.

Worked Example: Average of the Active Levels Versus Control

A prespecified question asks whether the average population response across the three active levels differs from the control mean. Define the contrast as the average active mean minus the control mean. In the order Control, Level 1, Level 2, Level 3, its coefficients are \((-1,\frac13,\frac13,\frac13)\), which sum to zero.

Estimate. Applying these coefficients to the sample means gives

\hat{L}=-20+\frac{23}{3}+\frac{25}{3}+\frac{28}{3} =\frac{16}{3}=5.333\text{ points}.

The observed average of the three active group means is \(76/3=25.333\) points, so the estimated active-average minus control difference is positive.

Standard error and test. For five observations per group,

SE(\hat{L}) =\sqrt{2\left(\frac{1}{5}+ \frac{(1/3)^2}{5}+\frac{(1/3)^2}{5}+\frac{(1/3)^2}{5}\right)} =\sqrt{2\left(\frac{1}{5}+\frac{1}{15}\right)} =\sqrt{\frac{8}{15}} =0.7303\text{ points}.

Testing \(H_0:L=0\) against \(H_a:L\ne0\) gives

t=\frac{16/3}{\sqrt{8/15}}=7.303,\qquad df=16.

The two-sided p-value is approximately \(0.0000018\). At the \(0.05\) significance level, the result is evidence that the average population response across the three active levels differs from the control mean. The estimated difference is positive, so the data point toward a higher active-level average. The p-value is the probability, assuming the null hypothesis and model, of a t statistic at least as extreme in either direction; it is not the probability that the null hypothesis is true.

Interval. Using \(t_{0.975,16}\approx2.120\), the 95% confidence interval is

\frac{16}{3}\pm2.120\sqrt{\frac{8}{15}} =5.333\pm1.548 =(3.785,\ 6.882)\text{ points}.

The interval estimates the average active-level population mean to be about 3.785 to 6.882 points higher than the control population mean. Its repeated-sampling interpretation is that this interval method would cover the true contrast in 95% of samples under the model; it does not mean there is a 95% probability that this fixed contrast lies in this particular interval.

Worked Example: Linear Trend Across Ordered Levels

A second prespecified question concerns whether the mean response follows a linear trend across equally spaced scores \(0,1,2,3\), assigned to Control, Level 1, Level 2, and Level 3. A convenient contrast for the slope per one-level increase uses coefficients proportional to the centered scores \((-1.5,-0.5,0.5,1.5)\). Dividing by their sum of squares, \(5\), gives \((-0.3,-0.1,0.1,0.3)\). They sum to zero.

Estimate. The estimated slope contrast is

\hat{L}=(-0.3)(20)+(-0.1)(23)+(0.1)(25)+(0.3)(28) =-6-2.3+2.5+8.4=2.600.

This contrast is expressed in points per one-level increase in the assigned score. It summarizes the linear component across these four means; it does not require the observed means to lie exactly on a straight line.

Standard error and test. The squared coefficients sum to \(0.09+0.01+0.01+0.09=0.20\), so

SE(\hat{L})=\sqrt{2\left(\frac{0.20}{5}\right)} =\sqrt{0.08}=0.2828\text{ points per level}.

For \(H_0:L=0\) against \(H_a:L\ne0\),

t=\frac{2.600}{0.2828}=9.192,\qquad df=16.

The two-sided p-value is less than \(0.000001\). Under the model, this is strong evidence against a zero linear trend in the specified direction-free test; the positive estimate indicates an increasing trend in the sample means. A 95% confidence interval is

2.600\pm2.120(0.2828) =2.600\pm0.5995 =(2.000,\ 3.200)\text{ points per level}.

The interval describes the linear contrast per one-step increase in the assigned scores. Since the group labels are ordered and equally spaced by stipulation, that slope summary is interpretable; it does not show that increasing a dose causes the response to rise unless the study design supports a causal conclusion.

Why the Standard Error Has This Form

The contrast standard error follows from the independent group means. Under the model, \(\bar{x}_i\) has variance \(\sigma^2/n_i\). Because observations in different groups are independent, the sample means are independent, and the variance of a weighted sum is the sum of the weighted variances:

\operatorname{Var}(\hat{L}) =\operatorname{Var}\left(\sum_i c_i\bar{x}_i\right) =\sum_i c_i^2\operatorname{Var}(\bar{x}_i) =\sigma^2\sum_i\frac{c_i^2}{n_i}.

Replacing the unknown \(\sigma^2\) by the pooled estimate \(MSE\) gives the stated estimated variance and its square root, the standard error. Under normal errors, \(\hat{L}\) is normally distributed, and the pooled within-group variance estimate is independent of the group means. Thus, under \(H_0:L=0\), dividing \(\hat{L}\) by its estimated standard error gives a t statistic with \(N-a\) degrees of freedom. This explains both the test and the t-based confidence interval.

Worked Example: Level 2 Versus Level 1

A third planned question compares Level 2 directly with Level 1. Define the contrast as Level 2 minus Level 1, with coefficients \((0,-1,1,0)\). This is a contrast because the coefficients sum to zero.

Estimate and standard error.

\hat{L}=0(20)-1(23)+1(25)+0(28)=2\text{ points}, \qquad SE(\hat{L})=\sqrt{2\left(\frac{1}{5}+\frac{1}{5}\right)} =\sqrt{0.8}=0.8944\text{ points}.

Test. The statistic for \(H_0:L=0\) against \(H_a:L\ne0\) is

t=\frac{2}{0.8944}=2.236,\qquad df=16.

The two-sided p-value is approximately \(0.040\). At the \(0.05\) level, this individual test provides evidence that the Level 2 population mean differs from the Level 1 population mean; the sample estimate indicates Level 2 is higher by 2 points.

Interval. The 95% confidence interval is

2\pm2.120(0.8944) =2\pm1.896 =(0.104,\ 3.896)\text{ points}.

This interval estimates the population mean at Level 2 minus the population mean at Level 1. It is narrowly above zero, consistent with the two-sided test at the 0.05 level. The inference is for this contrast considered individually; if conclusions are required to have simultaneous error control across a defined collection of comparisons, an appropriate multiple-comparison procedure is needed.

Common Pitfall and Reporting

A frequent error is to calculate the contrast standard error by adding the coefficients, or by using only the number of groups. The correct expression uses the squared coefficients divided by their respective group sizes, multiplied by \(MSE\). For example, in the active-average comparison, the three active coefficients each contribute \((1/3)^2/5=1/45\), not \(1/9\) by itself. The full variance estimate is \(2(1/5+3/45)=8/15\), so its standard error is \(\sqrt{8/15}=0.7303\) points.

Another mistake is to describe a contrast without preserving its direction. “The active treatments differ from control” loses the definition of the coefficients. Reporting “the average of the three active levels minus control was estimated as 5.333 points” states what was compared and makes the positive sign meaningful. A clear report also gives the standard error or interval, the test result with degrees of freedom and p-value when testing, and the model conditions that support the inference.

Key takeaway: Write the scientific comparison as coefficients first; then calculate \(\hat{L}\), use \(MSE\) and the group sizes for its standard error, and interpret the resulting test or interval in the direction and units defined by those coefficients.

Check Your Understanding

Use the definitions and calculations in this workshop to answer the following questions.

For the average of three active means minus control, why do the coefficients \((-1,\frac13,\frac13,\frac13)\) satisfy the contrast condition?
If the active-average estimate is \(16/3\) points and its standard error is \(\sqrt{8/15}\) points, write the two-sided t statistic and its degrees of freedom.
In the ordered-level example, what does the positive slope contrast estimate of 2.600 points per level describe, and what does it not establish by itself?
Explain why the contrast variance uses squared coefficients and why unequal group sizes would affect the standard error.
Why should the Level 2 versus Level 1 result not automatically be described as simultaneous evidence for every comparison among the four groups?