Why Compare Treatments With a Shared Control?
Suppose a study compares several doses of a new treatment with a standard control. The scientific question may be whether each dose changes the mean response relative to control, not whether every dose differs from every other dose. Testing each treatment-control difference separately at level \(\alpha\) does not, in general, keep the probability of one or more false rejections across the whole set at \(\alpha\). Dunnett’s procedure addresses that family of comparisons directly.
The key feature is that every comparison uses the same control group. Consequently, the estimated differences are statistically dependent: an unusually high or low control mean affects all the differences together. Dunnett’s method accounts for this dependence to provide adjusted p-values or simultaneous confidence intervals, usually with more precision than a general method that protects a much larger family.
Model, Comparisons, and Test Statistics
Consider a one-way ANOVA with one control group and \(m\) treatment groups. Let \(\mu_0\) be the population mean for the control and \(\mu_i\) the population mean for treatment \(i\), for \(i=1,\ldots,m\). The target contrasts are \(\mu_i-\mu_0\). A positive difference means the treatment population mean is higher than the control population mean; a negative difference means it is lower.
The usual model assumes independent observations, normally distributed errors within each group, and a common population variance \(\sigma^2\) across groups. The group observations are independent across individuals, and the response is measured on a scale for which group means are meaningful. The pooled error variance estimate is the ANOVA mean square error, \(MSE\), with error degrees of freedom \(df\). These are the one-way ANOVA assumptions; the design and data must make them reasonable before relying on the procedure.
For treatment \(i\), let \(\bar{x}_i\) and \(n_i\) be its sample mean and sample size, and let \(\bar{x}_0\) and \(n_0\) be the control sample mean and sample size. The estimated treatment-control difference and its standard error are
The test statistic is \(t_i=(\bar{x}_i-\bar{x}_0)/SE_i\). For a two-sided comparison, the null hypothesis is \(H_0:\mu_i-\mu_0=0\), against \(H_a:\mu_i-\mu_0\ne 0\). Dunnett’s adjustment evaluates the evidence for each contrast relative to the distribution of the most extreme statistic in the whole family, rather than treating each statistic as if it were the only one examined.
Why the Test Statistics Are Dependent
The shared control explains the dependence Dunnett’s method models. Under the common-variance model, the estimated difference for treatment \(i\) is \(\bar{x}_i-\bar{x}_0\). For two distinct treatment groups \(i\) and \(j\), their treatment means are independent, but both differences contain \(-\bar{x}_0\). Thus the covariance of the differences is the variance of the control mean.
Dividing this covariance by the two standard deviations gives the correlation:
For example, if all groups have the same size, this correlation is \(1/2\), not zero. The full test-statistic distribution is a correlated multivariate \(t\) distribution with the ANOVA error degrees of freedom. Dunnett’s calculation uses that joint distribution.
Worked Example: Three Treatments Compared With One Control
Worked Example: Three Treatments Compared With One Control
In a hypothetical laboratory experiment, a response is measured for a control and three treatment groups. Each group has \(n=10\) independent observations. The sample means are 20.0, 25.5, 22.0, and 17.5 response units, respectively; the pooled ANOVA error mean square is \(MSE=16.0\), with \(df=36\). Assume the design and response make the one-way ANOVA assumptions reasonable. Calculate the comparisons and interpret the Dunnett results.
The standard error is the same for all three contrasts because the group sizes are equal:
The treatment-control differences and test statistics are:
Suppose a correctly configured, two-sided Dunnett calculation for these three contrasts and \(df=36\) gives an adjusted p-value of approximately 0.009 for Treatment 1. This value is plausible for \(t_1=3.074\): the unadjusted two-sided \(t\)-tail probability is about 0.004, and the union bound says the Dunnett adjusted p-value cannot exceed three times that probability, about 0.012. At significance level \(\alpha=0.05\), Treatment 1 has evidence of a difference from control after accounting for the three comparisons. The estimated mean response is 5.5 units higher, but this is an estimated difference, not proof that treatment causes an increase in other settings.
The other two comparisons have smaller absolute test statistics. Their adjusted p-values must be at least as large as the adjusted p-value for Treatment 1, and they do not provide comparably strong evidence against their respective null hypotheses. The adjustment applies to the family of these three comparisons, not to every possible contrast among all four groups.
Simultaneous Confidence Intervals
Dunnett’s procedure can also produce confidence intervals that cover all \(m\) treatment-control differences simultaneously with confidence level \(1-\alpha\). Let \(c\) be the appropriate Dunnett critical value for the chosen family size, degrees of freedom, design, and two-sided or one-sided alternative. Each two-sided interval has the form
Unlike separate ordinary \(t\) intervals, these intervals use a critical value calibrated to achieve joint coverage for the whole set. A 95% simultaneous procedure has, under the model, a 95% probability of covering every true treatment-control difference in the family over repeated samples. It does not mean there is a 95% probability that a fixed parameter lies in a particular computed interval.
For the balanced example, suppose the appropriate software reports a two-sided Dunnett critical value of \(c=2.45\), rounded to three significant figures. Treatment 1’s simultaneous interval is
Because this interval excludes zero, it agrees with the two-sided test at the corresponding familywise level: the data indicate that Treatment 1’s mean response differs from the control mean. The interval estimates the difference as between about 1.12 and 9.88 response units higher. An interval containing zero would not establish a difference at that familywise level.
Worked Example: Unequal Group Sizes
Worked Example: Unequal Group Sizes
In a second hypothetical experiment, a control group has \(n_0=12\), and two treatments have \(n_1=8\) and \(n_2=15\). The sample means are 100.0 for control and 106.0 and 102.6 for the treatments. The one-way ANOVA gives \(MSE=25.0\) and \(df=32\). Assume independent observations, approximately normal within-group errors, and a common variance are reasonable for this design.
The first contrast has standard error and test statistic
For the second contrast,
The two estimated differences share the same control mean. Their correlation under the model is
Thus the test statistics are not independent. A Dunnett calculation for two comparisons, \(df=32\), and this unequal-size design accounts for their dependence. It would be incorrect to use a critical value for balanced groups without checking that it applies. A software package or an appropriate Dunnett table must be configured for the right number of treatments, degrees of freedom, group-size structure, and tail choice.
Worked Example: Reading an Adjusted p-Value
Worked Example: Reading an Adjusted p-Value
Suppose a hypothetical study has four treatments compared with one control. For one treatment, the observed absolute test statistic is 3.20 with 40 error degrees of freedom. A correctly configured two-sided Dunnett analysis reports an adjusted p-value of 0.008. Interpret this result at \(\alpha=0.05\).
The adjusted p-value is the probability, assuming all four treatment-control null hypotheses are true and the model holds, of obtaining at least one absolute Dunnett test statistic at least as large as 3.20 in this four-comparison family. It is not the probability that this particular null hypothesis is true. Since \(0.008<0.05\), reject the null for this treatment-control contrast under the procedure. In context, the data provide evidence that this treatment’s population mean differs from the control mean, while familywise error is controlled across the four specified comparisons.
The adjustment is tied to the declared family. If the scientific question instead included comparisons among treatments themselves, those additional comparisons would require a procedure that protects that broader family. Dunnett’s procedure does not automatically cover them.
Applying and Reporting Dunnett’s Procedure
Identify the one control and list every treatment-control contrast that belongs to the analysis. Decide whether each comparison is two-sided or whether a direction was specified in advance.
Verify that the one-way ANOVA conditions are reasonable, then use its \(MSE\) and error \(df\). Do not replace the common pooled variance with a different variance estimate for each contrast.
For each treatment, calculate \(\bar{x}_i-\bar{x}_0\) and \(\sqrt{MSE(1/n_i+1/n_0)}\), then divide to obtain its test statistic.
Obtain adjusted p-values or simultaneous intervals for the specified family using the correct degrees of freedom, group sizes, and tail choice.
State which treatments differ from control, give estimated differences and units, and name the familywise confidence level or significance level used.
Common Mistakes and Why Dunnett Matters
Dunnett’s procedure is useful when the scientific goal is specifically to screen several treatments against a shared benchmark. It uses the family definition efficiently: unlike all-pairs procedures, it does not spend error protection on treatment-treatment comparisons that are not part of the question. This does not make it universally preferable; it is appropriate when treatment-versus-control comparisons are the intended family.
- Using unadjusted p-values for every treatment. Testing each contrast at \(\alpha\) and reporting them as a family ignores the increased chance of at least one false rejection. Use the Dunnett adjusted p-values or its simultaneous intervals.
- Using an all-pairs method for the wrong question. If only treatment-control differences are of interest, an all-pairs method protects comparisons that were not requested and can be unnecessarily conservative.
- Assuming the comparisons are independent. Their shared control creates dependence. Dunnett’s critical values and adjusted p-values account for that joint structure.
- Changing the family after seeing results. Adding or removing comparisons changes the multiplicity adjustment. Define the family based on the scientific question, not on which results look promising.
- Calling a non-significant result proof of equality. A large adjusted p-value means the data do not provide sufficient evidence of a difference at the chosen familywise level; it does not demonstrate that the means are identical.
Check Your Understanding
Try these questions before reviewing the worked examples.
- A study compares five doses with a placebo, and only dose-versus-placebo effects are of interest. Why is Dunnett’s procedure suited to this family?
- Explain why two estimated treatment-control differences are correlated even when the treatment groups themselves are independent.
- For a treatment with \(n_i=10\), a control with \(n_0=20\), and \(MSE=9\), calculate the standard error of the treatment-control difference.
- What does a two-sided Dunnett-adjusted p-value of 0.03 mean when four treatment-control comparisons are in the family?
- A simultaneous 95% Dunnett interval for a mean difference includes zero. What conclusion is justified, and what conclusion would go too far?