Why the Joint Distribution Matters
When several candidate treatments are compared with a control, their test statistics are generally not independent. In particular, comparisons that share a control group share sampling variability. A single-step maxT procedure uses the joint distribution of those statistics to set one critical value for the largest statistic. The goal is to test each candidate while controlling the probability of making one or more false rejections across the family.
The earlier material on Dunnett adjustment treats the shared-control setting. Here the central idea is the distribution of the maximum itself: once that distribution is specified, the critical value follows directly. The method can also cover correlated treatment statistics arising from other prespecified designs, provided their joint distribution is correctly calibrated.
For candidate arm \(j\), consider \(H_{0j}:\theta_j\leq 0\) against \(H_{aj}:\theta_j>0\), where \(\theta_j\) is the treatment effect relative to the control. Let \(\hat{\theta}_j\) estimate that effect and let \(SE_j\) be its standard error. The statistic is \(Z_j=\hat{\theta}_j/SE_j\). The procedure rejects \(H_{0j}\) when \(Z_j>c_\alpha\). The null boundary \(\theta_j=0\) is the least favorable case for this upper-tail test; effects below zero make rejection less likely under the stated model.
Model and Critical-Value Derivation
The basic calibration assumes that, under the complete null, the vector \(Z=(Z_1,\ldots,Z_m)^\mathsf{T}\) has a multivariate normal distribution with mean zero and a prespecified correlation matrix \(R\). Each statistic is standardized to have variance one. Thus \(Z\sim N_m(0,R)\). In a finite-sample analysis, this can be an exact model; in a large-sample analysis, it is an approximation whose accuracy must be supported by the design and data.
A critical assumption for strong familywise error control is subset pivotality: the joint distribution of the statistics corresponding to the true null hypotheses does not depend on which other hypotheses are false. In this setting, that means the true-null statistics have the same centered joint distribution and correlation structure used for calibration. If this condition fails, control under the complete null alone does not automatically establish control for every configuration of true and false hypotheses.
For a specified \(R\), the joint cumulative distribution function gives \(P(M\leq c)=P(Z_1\leq c,\ldots,Z_m\leq c)\). The maxT cutoff is the value at which this probability reaches \(1-\alpha\). In general, the probability is evaluated using multivariate normal numerical integration; there need not be a closed-form expression.
An especially useful case is an equicorrelation model: every pair of distinct statistics has the same correlation \(\rho\). For \(0\leq\rho<1\), write \(Z_j=\sqrt{\rho}\,U+\sqrt{1-\rho}\,\varepsilon_j\), where \(U,\varepsilon_1,\ldots,\varepsilon_m\) are independent standard normal random variables. Conditional on \(U=u\), the \(Z_j\) are independent, each with mean \(\sqrt{\rho}\,u\) and variance \(1-\rho\). Therefore the conditional probability that all are at most \(c\) is the \(m\)th power of the corresponding single-statistic probability. Averaging over \(U\) yields:
This derivation identifies why the correlation matters: the common component \(U\) makes large values tend to occur together. The joint tail probability is not generally the sum of the marginal tail probabilities, nor is it the probability obtained by treating the tests as independent.
Worked Examples
Worked Example: Correlation From a Shared Control
Suppose, hypothetically, three experimental arms and one control each have 64 participants. Assume independent normally distributed outcomes with a common standard deviation of 12 outcome units. For each arm, the estimated effect is the experimental-arm mean minus the control mean. Find the standard error and the correlation between any two standardized treatment-control statistics.
The estimated difference for arm \(j\) is \(\hat{\theta}_j=\bar{X}_j-\bar{X}_0\). Its variance is \(12^2/64+12^2/64=4.5\) outcome-units squared, so \(SE_j=\sqrt{4.5}=2.121\) outcome units. For two different arms \(j\) and \(k\), the experimental-arm means are independent of each other and of the control mean. Their differences share the control mean, so
Each difference has variance 4.5, giving correlation \(2.25/\sqrt{4.5\times4.5}=0.5\). Standardizing by the common standard error does not change this correlation. Under the normal, equal-variance assumptions, the three statistics therefore have an equicorrelated joint normal null distribution with \(\rho=0.5\). The covariance calculation explains why an independence-based cutoff would not describe this design.
Worked Example: Finding the Three-Arm MaxT Cutoff
Continue with the hypothetical three-arm design, and use a one-sided familywise significance level of \(\alpha=0.05\). Here \(m=3\) and \(\rho=0.5\). Substitution into the equicorrelated-normal formula gives the equation to solve:
Numerical integration followed by one-dimensional root finding gives \(c_{0.05}\approx 2.06\). The calculation uses the full joint distribution, rather than a separate 0.05 test for each arm. For comparison, a Bonferroni cutoff solves \(\Phi(c)=1-0.05/3\), which gives approximately \(2.128\). Bonferroni does not use the correlation and is more conservative here. The maxT value is not a universal three-test cutoff: it depends on the assumed joint distribution as well as on \(m\) and \(\alpha\).
Worked Example: Testing Hypothetical Arm Statistics
Suppose a confirmatory analysis of the same hypothetical design produces standardized statistics \(Z_1=2.20\), \(Z_2=1.95\), and \(Z_3=1.40\). Use \(c_{0.05}=2.06\) to decide which one-sided hypotheses to reject.
Compare each statistic with the common maxT cutoff. Since \(2.20>2.06\), reject \(H_{01}\). Since \(1.95\leq2.06\) and \(1.40\leq2.06\), do not reject \(H_{02}\) or \(H_{03}\). If the standard errors are each 2.121 outcome units, the corresponding estimated effects are \(2.20(2.121)=4.666\), \(1.95(2.121)=4.136\), and \(1.40(2.121)=2.969\) outcome units. The decision is based on the standardized statistics, not on the largest estimated effect alone. The maxT calibration limits the probability of one or more false rejections across the three candidate-arm tests to at most 0.05 under its assumptions.
Implementation, Diagnostics, and Alternatives
A practical analysis begins by defining the family of candidate hypotheses and the direction of testing. It then specifies the joint distribution of their test statistics under the null, including the correlation matrix, and calculates the appropriate upper-tail maximum quantile. Each statistic is compared with the same cutoff. If the design or analysis produces a multivariate \(t\) distribution rather than a multivariate normal distribution, the reference distribution and critical value must use that \(t\) model instead; substituting a normal cutoff without justification can miscalibrate error.
List the candidate-arm null hypotheses and confirm whether the tests are one-sided upper-tail tests.
State the marginal distributions and the correlation matrix for the standardized statistics, and justify the assumptions from the trial design and analysis model.
Evaluate \(P(M\leq c)\) under that joint model and solve for \(c\) such that the probability is \(1-\alpha\).
Reject each candidate-arm null for which \(Z_j>c\), and report the familywise level and model assumptions alongside the decisions.
Useful diagnostics include checking that the assumed covariance agrees with the planned allocation, shared-control structure, and analysis model; verifying that each statistic has the stated null marginal distribution; and checking numerical integration by confirming that the computed joint probability at the reported cutoff is close to \(1-\alpha\). Sensitivity calculations across plausible correlation values can reveal whether small covariance misspecification materially changes the cutoff. If the covariance is estimated imprecisely, treating the estimate as known can understate uncertainty and compromise nominal error control.
The calibration also needs to reflect all statistics that could lead to a confirmatory rejection. If arms are selected using the same outcome data, calculating a cutoff only for the observed winners and treating their number as fixed can ignore selection-induced multiplicity. A safer single-step approach is to calibrate the maximum over the original candidate family. More generally, if adaptation changes the distribution of the test statistics, the reference model must represent that design; a fixed-design covariance matrix cannot simply be assumed valid.
| Approach | How it sets the threshold | Main trade-off |
|---|---|---|
| Bonferroni | Uses a marginal level of \(\alpha/m\) for each test. | Simple and valid without modeling dependence, but may be conservative. |
| MaxT | Calibrates the maximum under the joint distribution of the statistics. | Uses correlation information, but depends on a valid joint model and accurate computation. |
| Holm step-down | Orders marginal p-values and applies sequential thresholds. | Provides strong control under general dependence, but is not the same single common-cutoff rule. |
| Closed testing | Tests intersection hypotheses to determine which elementary hypotheses can be rejected. | Can exploit structured tests, but may require more calculations than a single-step maxT analysis. |
MaxT is limited by its calibration assumptions. A normal approximation may be poor with very small samples or problematic outcome distributions. Correlation estimates based on unstable variance components can also lead to an unreliable cutoff. For two-sided hypotheses, the relevant family statistic is typically \(\max_j |Z_j|\), with a separately calibrated cutoff; the one-sided cutoff derived here must not be reused without recalibration. These are modeling and implementation issues, not reasons to ignore dependence: they indicate that the joint reference distribution needs checking or a more suitable method.
Common Mistake
A common mistake is to treat the maxT cutoff as the ordinary single-test critical value because every statistic is standardized. Standardization gives each statistic a common marginal scale; it does not make the maximum have the same distribution as one statistic. Another mistake is to assume that positive correlation means no multiplicity adjustment is needed. Correlation changes the maximum’s distribution, but it does not remove the possibility of a false rejection across several tests. The correction is to calibrate the maximum under the justified joint null distribution, then compare all relevant statistics with that familywise cutoff.
Check Your Understanding
Try each problem, then compare your reasoning with the solutions below.
- For two standardized treatment statistics with correlation 0, what is the defining equation for a one-sided maxT cutoff at level \(\alpha\)?
- In the shared-control example, explain why the treatment-control statistics are correlated even though the experimental arms are independent.
- Three candidate-arm statistics have a maxT cutoff of 2.06. Which hypotheses are rejected if the observed statistics are 1.88, 2.09, and 2.01?
- Why does calibrating a cutoff only for the selected arms risk invalid error control when selection and testing use the same data?
Solutions. (1) Under independence, \(P(M\leq c)=\Phi(c)^2\). Thus the cutoff solves \(\Phi(c)^2=1-\alpha\), or \(c=\Phi^{-1}(\sqrt{1-\alpha})\). (2) Each estimated difference subtracts the same control mean. The covariance between two differences is the variance of that shared control mean, which is positive. (3) Only the second hypothesis is rejected because \(2.09>2.06\); the other two statistics do not exceed the cutoff. (4) The selected statistics are chosen partly because their observed values are large. Treating the selected set as if it had been fixed in advance ignores this data-dependent selection and can make the probability of at least one false rejection exceed the target. A calibration over the original candidate family avoids that particular omission, subject to the joint-model assumptions.