Dunnett Adjustment For Shared-Control Comparisons

Why Shared-Control Comparisons Need a Joint Adjustment

When several experimental treatments are compared with one control, the treatment-control estimates are not independent: every contrast contains the same control estimate. A procedure that treats them as independent can miscalculate the chance of at least one false positive. Dunnett adjustment uses the joint distribution of these contrasts to control the familywise error rate, often with less conservatism than a general-purpose adjustment.

The focus here is the standard balanced, shared-control setting. Suppose there are \(k\) experimental arms and one control arm, with \(n\) independent participants in each arm. For a continuous outcome, let the arm means be \(\bar{X}_i\), where \(i=0\) denotes control and \(i=1,\ldots,k\) denotes treatments. Assume independent normally distributed outcomes with a common variance \(\sigma^2\). First suppose \(\sigma\) is known. These assumptions give a direct multivariate normal calculation; the unknown-variance version is described below.

Definition: For treatment \(i\), the contrast is \(D_i=\bar{X}_i-\bar{X}_0\). Its standardized statistic is \(Z_i=D_i/SE(D_i)\). A Dunnett critical value \(c\) is chosen so that the probability of at least one statistic crossing the relevant one-sided or two-sided cutoff under the joint null distribution is \(\alpha\). The resulting single-step tests compare every treatment with the shared control using the same adjusted cutoff.

Where the Correlation Comes From

Each arm mean has variance \(\sigma^2/n\). Because treatment arms are independent of one another and of control, two different contrasts share only the control term. For \(i\ne j\),

\operatorname{Var}(D_i)=\frac{2\sigma^2}{n}, \qquad \operatorname{Cov}(D_i,D_j)=\operatorname{Var}(\bar{X}_0)=\frac{\sigma^2}{n}, \qquad \operatorname{Corr}(D_i,D_j)=\frac{1}{2}.

Standardizing does not change this correlation. Thus the vector of \(k\) statistics has a multivariate normal distribution with zero means under the joint null and a correlation matrix with 1s on the diagonal and \(1/2\) elsewhere. Positive correlation means that unusually high treatment-control statistics tend to occur together more often than they would if the contrasts were independent.

For a one-sided superiority test, the hypotheses for treatment \(i\) are \(H_{0i}:\mu_i-\mu_0\leq 0\) versus \(H_{ai}:\mu_i-\mu_0>0\). Reject \(H_{0i}\) when \(Z_i>c\). For two-sided tests, use \(H_{0i}:\mu_i-\mu_0=0\) versus \(H_{ai}:\mu_i-\mu_0\ne 0\), and reject when \(|Z_i|>c\). The two-sided cutoff is not obtained by simply doubling the one-sided cutoff: it is calibrated to the probability that any statistic crosses either tail.

Calibration rule: For one-sided tests, choose \(c\) to satisfy \(P(\max_{1\leq i\leq k} Z_i>c)=\alpha\). For two-sided tests, choose \(c\) to satisfy \(P(\max_{1\leq i\leq k}|Z_i|>c)=\alpha\). These are familywise error probabilities under the joint null.

Deriving the Dunnett Cutoff

For equal pairwise correlation \(\rho=1/2\), a convenient representation of the standardized statistics is \(Z_i=\sqrt{\rho}\,U+\sqrt{1-\rho}\,\varepsilon_i\), where \(U,\varepsilon_1,\ldots,\varepsilon_k\) are independent standard normal variables. Conditional on \(U=u\), the \(Z_i\) are independent normals with mean \(\sqrt{\rho}u\) and variance \(1-\rho\). This conditional independence turns the joint probability into a one-dimensional integral.

P(\max_i Z_i\leq c) = \int_{-\infty}^{\infty} \phi(u) \left[ \Phi\left(\frac{c-\sqrt{\rho}u}{\sqrt{1-\rho}}\right) \right]^k\,du.

Here \(\phi\) and \(\Phi\) are the standard normal density and distribution functions. The one-sided critical value solves \(P(\max_i Z_i\leq c)=1-\alpha\). For two-sided tests, conditional on \(U=u\), each statistic lies between \(-c\) and \(c\) with probability equal to the difference of two normal distribution functions. Therefore,

P(\max_i |Z_i|\leq c) = \int_{-\infty}^{\infty} \phi(u) \left[ \Phi\left(\frac{c-\sqrt{\rho}u}{\sqrt{1-\rho}}\right) - \Phi\left(\frac{-c-\sqrt{\rho}u}{\sqrt{1-\rho}}\right) \right]^k\,du.

These integrals are evaluated numerically or by using a Dunnett table or statistical software. The derivation shows why the cutoff depends on the number of comparisons, their correlation, whether the test is one- or two-sided, and the chosen \(\alpha\).

Worked Example: Finding Cutoffs for Three Comparisons

Suppose three treatments are each compared with a common control, all four arms have equal sample sizes, and the outcomes meet the normal, common-variance assumptions. For a one-sided familywise level \(\alpha=0.05\), the correlation is \(\rho=0.5\). Substituting \(k=3\) and \(\rho=0.5\) into the one-sided integral, a numerical solution to \(P(\max_i Z_i\leq c)=0.95\) is approximately \(c=2.04\).

For a two-sided familywise level of 0.05, solve the two-sided integral instead. Its approximate solution is \(c=2.34\). The two-sided rejection rule is \(|Z_i|>2.34\) for at least one treatment comparison. These values are approximate; a software implementation should match the assumed tail convention, correlation, degrees of freedom, and number of arms.

For comparison, separate Bonferroni cutoffs would be approximately 2.13 for three one-sided tests \(\bigl(\Phi^{-1}(1-0.05/3)\bigr)\) and 2.39 for three two-sided tests \(\bigl(\Phi^{-1}(1-0.05/6)\bigr)\). The Dunnett cutoffs are lower here because the shared-control statistics are positively correlated, not independent.

Applying the Adjustment to Treatment Effects

Worked Example: Testing Three Treatment Effects

Consider hypothetical normally distributed outcomes from four balanced arms, with \(n=40\) participants per arm. Suppose the known common standard deviation is \(10\) outcome units. The observed means are 100 units for control and 106, 104, and 102 units for treatments 1, 2, and 3. The estimated differences are therefore 6, 4, and 2 units. The standard error for each difference is

SE(D_i)=\sqrt{\frac{10^2}{40}+\frac{10^2}{40}} =\sqrt{5}=2.236\text{ units}.

The standardized statistics are \(Z_1=6/2.236=2.683\), \(Z_2=4/2.236=1.789\), and \(Z_3=2/2.236=0.894\). For one-sided superiority tests at familywise \(\alpha=0.05\), compare each with the three-comparison Dunnett cutoff \(2.04\). Treatment 1 exceeds the cutoff; treatments 2 and 3 do not. Under the stated model, there is familywise-adjusted evidence that treatment 1 has a higher mean outcome than control. This does not establish that treatment 1 is clinically beneficial: the clinical meaning of a 6-unit difference must be assessed in context.

If the prespecified questions were two-sided instead, compare the absolute statistics with 2.34. Only treatment 1 exceeds that cutoff as well. The direction and tail convention should be set by the scientific question before examining the results.

Selection and the Definition of the Testing Family

Dunnett adjustment controls error for a specified family of treatment-control hypotheses. That family matters when arms have been selected. If a subset is selected using interim data that are independent of the final confirmatory data, then, conditional on the selected subset, the final statistics retain their ordinary joint distribution. A Dunnett cutoff for the number of retained arms can then control the familywise error rate among those retained comparisons, provided the model and design assumptions hold.

If arms are selected using the same statistics that will be tested, applying a cutoff recalculated only for the winners is generally not valid: the winners tend to have unusually large observed effects, and the smaller selected family does not account for this selection. One safe approach, when the original fixed-sample joint model applies, is to retain the cutoff calibrated for all candidate arms. Any false rejection among selected arms is then also a rejection among the full candidate family, whose error probability is controlled. When interim data are reused in the final analysis, there are repeated looks, or the final statistics have otherwise been altered by the adaptation, the ordinary fixed-sample Dunnett distribution may no longer apply; the design needs an analysis that accounts for those features.

Worked Example: Two Arms Selected Before Independent Confirmation

Suppose four candidate treatments are screened at an interim stage, and the two retained arms are chosen using only interim data. A separate confirmatory sample is then collected, with equal sample sizes in the two retained arms and their shared control. Assume independent normal outcomes with a common known variance, and prespecify one-sided superiority tests at familywise \(\alpha=0.05\).

Because the confirmatory observations are independent of the selection data, conditioning on which two arms were retained does not change the confirmatory statistics’ distribution. For two retained comparisons, the shared-control correlation remains \(0.5\). The approximate one-sided two-comparison Dunnett cutoff is 1.92. If the confirmatory statistics are 2.10 and 1.50, only the first exceeds 1.92, so only that treatment rejects its null hypothesis at the stated familywise level for the retained family.

This conclusion depends on the independent confirmatory sample. If the same outcomes or statistics had determined which two arms were retained and tested, the two-comparison cutoff would ignore winner selection. In that setting, a cutoff calibrated for the full four-arm family can protect the original family under the fixed-sample joint model; simply recalibrating for the two observed winners cannot.

Unknown Variance, Alternatives, and Diagnostics

When the common variance is unknown, it is usually estimated by pooling within-arm variation. With equal \(n\) in all \(k+1\) arms, the pooled variance is

s_p^2= \frac{\sum_{i=0}^{k}(n-1)s_i^2}{(k+1)(n-1)}, \qquad t_i= \frac{\bar{X}_i-\bar{X}_0}{s_p\sqrt{2/n}}.

Under independent normal observations with a common variance, the statistics have a joint multivariate \(t\) distribution with pairwise correlation \(0.5\) and \(df=(k+1)(n-1)\). Use the corresponding Dunnett \(t\) cutoff, not the normal cutoff; it depends on the degrees of freedom and is typically larger for finite samples.

Dunnett is specialized to comparisons against a common control. Bonferroni is simpler and does not require the correlation structure, but can be more conservative. Holm’s step-down procedure also controls familywise error under general dependence and may be useful for a broader set of hypotheses, but it does not use the shared-control correlation to calculate a single Dunnett cutoff. These procedures answer related but not identical testing questions.

Check the analysis before interpreting the adjusted results. Unequal allocation changes the contrast correlations; unequal variances or non-normal small-sample outcomes can undermine the standard model; and an incorrect arm count or tail choice changes the cutoff. In particular, do not use balanced-design values such as \(\rho=0.5\) when sample sizes differ. If the design or data structure departs from the assumptions, use a method whose joint distribution matches that design rather than treating a tabulated Dunnett value as universal.

Common pitfall: “Only two treatments survived selection, so I can use the less stringent two-comparison Dunnett cutoff on the same statistics that selected them.” This overlooks that the selected statistics are winners by construction. The correction is to account for the selection: use independent confirmatory data and calibrate to the retained family, or, under the applicable fixed-sample joint model, use a cutoff for the full candidate family. Reused interim data or additional adaptation can require a different calibration.

Key takeaway: Dunnett adjustment controls the chance of one or more false treatment-versus-control rejections by calibrating a cutoff to the joint distribution of correlated contrasts. Its efficiency depends on modeling that correlation correctly; its validity after selection depends on how the selection relates to the data used for testing.

Check Your Understanding

Answer each question, then compare with the solutions.

In a balanced design with a common control and equal arm sizes, why is the correlation between two standardized treatment-control contrasts positive?
For three one-sided comparisons at familywise \(\alpha=0.05\), the approximate Dunnett cutoff is 2.04. A treatment statistic is 2.00. Does it reject?
Why is the two-sided Dunnett cutoff not obtained by using the one-sided cutoff unchanged?
Two arms are selected using interim data and tested using an independent confirmatory sample. What makes a cutoff for the retained family appropriate?
What is the main error in applying a two-comparison cutoff to two winners selected and tested using the same statistics?

Solutions: (1) Both contrasts subtract the same control mean, so their covariance is the control mean’s variance, \(\sigma^2/n>0\); standardizing gives correlation \(1/2\). (2) No. Since \(2.00<2.04\), the statistic does not cross the adjusted one-sided cutoff. (3) Two-sided testing counts extreme values in both tails. Its cutoff must make the probability of any \(|Z_i|\) exceeding the cutoff equal to \(\alpha\), using the two-sided joint probability. (4) Conditional on which arms were selected, the independent confirmatory statistics retain their prespecified joint distribution, so the retained family can be calibrated using its number of comparisons and shared-control correlation. (5) The cutoff ignores selection: the tested statistics were chosen for being large. The analysis must account for the full candidate family or use independent confirmation; the selected-family cutoff alone does not resolve same-data selection.