Why the Contrast Family Must Be Chosen First
A repeated-measures study can support many comparisons: selected treatment occasions, every pair of occasions, polynomial trends, or contrasts within particular groups. Controlling error for a short list of prespecified comparisons does not automatically control error for every comparison examined afterward. The first step in multiplicity control is therefore to say exactly which contrasts belong to the inferential family.
Let \(Y_i\) be the \(p\)-component response vector for subject \(i\), with mean vector \(\mu\), and suppose the \(n\) subject vectors are independent and identically distributed. A within-subject contrast has the form \(c'\mu\), where \(c\) is a \(p\)-component vector whose entries sum to zero. Stack \(m\) contrasts as the rows of an \(m\times p\) matrix \(C\). The contrast mean vector is \(\eta=C\mu\), and the subject-level contrast scores are \(Z_i=CY_i\). Each contrast is thus an ordinary mean of the corresponding subject-level scores, while all contrasts together form a possibly correlated vector.
The family is determined by the scientific claim, not simply by the number of rows in a matrix used for computation. For example, a study might identify two contrasts as primary in advance, while also examining all pairwise occasion differences as exploratory analyses. Those are different families and call for different interpretations of the error guarantee.
Bonferroni Simultaneous Intervals
For contrast \(j\), let \(Z_{ij}=c_j'Y_i\), let \(\bar Z_j\) be its sample mean, and let \(s_j\) be its sample standard deviation. If the contrast scores are independent across subjects and normally distributed with positive variance, then for \(n\ge 2\), \((\bar Z_j-\eta_j)/(s_j/\sqrt n)\) has a \(t\) distribution with \(n-1\) degrees of freedom. Construct a two-sided interval for each of the \(m\) means using marginal noncoverage probability \(\alpha/m\):
Here \(t_{\nu,u}\) denotes the \(u\)-quantile of a \(t\) distribution with \(\nu\) degrees of freedom. The Bonferroni method does not require the contrast scores to be independent of one another. They may be highly correlated because they were measured on the same subjects.
Proof. Let \(A_j\) be the event that interval \(j\) misses \(\eta_j\). The event that at least one interval fails is \(\bigcup_{j=1}^m A_j\). The union bound gives
No independence assumption among the \(A_j\) is used. For positive-variance normal scores, the stated \(t\) pivot gives \(P(A_j)=\alpha/m\). The union bound may nevertheless be conservative, particularly when the contrasts are strongly related. \(\square\)
Worked Example: Bonferroni Intervals for Three Occasion Comparisons
Ten subjects are measured at three occasions. The family consists of all three pairwise mean differences. Suppose the observed means for the differences are \(2.1\) for occasion 1 minus 2, \(1.4\) for occasion 2 minus 3, and \(3.5\) for occasion 1 minus 3. The first two difference scores have sample standard deviations \(1.2\) and \(1.0\), and their sample covariance is zero. Since the third difference is the sum of the first two, its sample standard deviation is \(\sqrt{1.2^2+1.0^2}=\sqrt{2.44}\). Use familywise level \(\alpha=0.05\).
There are \(m=3\) intervals, so the common critical value is \(t_{9,\,1-0.05/(2\cdot3)}=t_{9,\,0.9916667}\), approximately \(2.93332\). The half-width for the first difference is \(2.93332(1.2/\sqrt{10})\approx1.113\). Its interval is \(2.1\pm1.113\), or approximately \((0.987,3.213)\).
For occasion 2 minus 3, the half-width is \(2.93332(1.0/\sqrt{10})\approx0.928\), giving \((0.472,2.328)\). For occasion 1 minus 3, the half-width is \(2.93332(\sqrt{2.44}/\sqrt{10})\approx1.449\), giving \((2.051,4.949)\). Together, these intervals cover all three pairwise differences with probability at least \(0.95\) under the normal-score assumptions. The shared subjects induce dependence among comparisons, but Bonferroni does not require that dependence to be estimated for its guarantee.
A Simultaneous Region for a Vector of Contrasts
Bonferroni assigns a marginal error budget to each member of the family. When the target is the whole vector of \(q\) contrast means, a different option is a multivariate confidence region. Let \(Z_i=CY_i\) now have dimension \(q\), and suppose the \(Z_i\) are independent and identically distributed as \(N_q(\eta,\Sigma_C)\), where \(\Sigma_C\) is positive definite. Define their sample mean \(\bar Z\) and unbiased sample covariance matrix \(S\). Assume \(n>q\).
Under these conditions, Hotelling's \(T^2\) distribution gives the exact pivot
Here \(F_{q,n-q}\) denotes an \(F\) distribution with the indicated degrees of freedom. Equivalently, the region
has coverage \(1-\alpha\). The result requires multivariate normality and a nonsingular contrast covariance; it is not a distribution-free guarantee. The region covers the contrast mean vector jointly, rather than assigning a separate marginal probability to each coordinate.
The ellipsoid also gives simultaneous intervals for any specified coordinates. Write \(K=q(n-1)F_{q,n-q;\,1-\alpha}/(n-q)\). For coordinate \(j\), the largest possible value of \(|\bar Z_j-\eta_j|\) among vectors in the ellipsoid is \(\sqrt{KS_{jj}/n}\). To see this, set \(d=\bar Z-\eta\). The region condition is \(d'S^{-1}d\leq K/n\). By the Cauchy–Schwarz inequality in the inner product defined by \(S^{-1}\),
Equality is attainable in the direction \(d\) proportional to \(Se_j\), so this bound is the exact projection of the ellipsoid onto coordinate \(j\). Consequently, intervals that jointly inherit the region’s coverage are
Because all these intervals are projections of the same region, the event that the region contains \(\eta\) implies that every projected interval contains its coordinate. Their joint coverage is therefore at least \(1-\alpha\). They are generally not the narrowest possible intervals for every coordinate separately; the guarantee is for the vector as a whole.
Worked Example: Projecting a Two-Contrast Region
Suppose \(n=12\) subjects provide two contrast scores per subject. Assume a bivariate normal model, and suppose the observed contrast mean vector and covariance matrix are \(\bar Z=(1.5,-0.5)'\) and \(S=\begin{pmatrix}4&0\\0&1\end{pmatrix}\). The family is the two-dimensional mean vector, and \(\alpha=0.05\).
Here \(q=2\), so \(F_{2,10;\,0.95}\approx4.1028\). The ellipsoid threshold is \(K=2(12-1)4.1028/(12-2)=2.2(4.1028)\approx9.0262\). Thus the joint region is \(12(\bar Z-\eta)'S^{-1}(\bar Z-\eta)\leq9.0262\).
The first projected interval has half-width \(\sqrt{9.0262(4)/12}\approx1.735\), giving approximately \((-0.235,3.235)\). The second has half-width \(\sqrt{9.0262(1)/12}\approx0.867\), giving approximately \((-1.367,0.367)\). Both intervals hold simultaneously with probability at least \(0.95\) under the stated model because they are projections of the one \(95\%\) confidence region.
Which Method Matches the Scientific Target?
The two procedures answer related but different questions. Bonferroni is convenient when a family consists of a manageable list of individual contrasts, perhaps selected in advance. It allows different contrast vectors and does not require a positive-definite covariance matrix for the entire vector. The Hotelling region is natural when the target is a specified vector of linearly independent contrasts and the multivariate normal model is credible. Its geometry uses the covariance among contrast scores directly.
The choice of family can matter more than the choice of critical value. If two contrasts were selected before data analysis, a two-contrast family may be justified for those claims. If the investigator searches through all pairwise differences and reports whichever appear most striking, the family must reflect that broader search for a familywise guarantee over the reported comparisons. A confidence region for one contrast subspace does not automatically protect contrasts outside that subspace.
| Feature | Bonferroni intervals | Hotelling contrast region |
|---|---|---|
| Target | A specified list of individual contrast means | A joint vector of linearly independent contrast means |
| Dependence among contrasts | No independence required; dependence is not used to set the bound | Represented through the sample covariance matrix |
| Key assumptions for exact stated method | Normal scores with positive variance for each ordinary \(t\) pivot | Multivariate normal contrast scores, positive-definite covariance, and \(n>q\) |
| Main guarantee | Familywise coverage at least \(1-\alpha\) | Joint coverage \(1-\alpha\) for the contrast vector |
Worked Example: A Zero-Variance Contrast in the Family
Suppose eight subjects are measured on three occasions, and occasions 1 and 2 are identical for every subject. The family contains \(c_1'\mu=\mu_1-\mu_2\) and \(c_2'\mu=\mu_2-\mu_3\). Then every observed score for \(c_1=(1,-1,0)'\) is zero, so its sample mean and sample standard deviation are both zero. Suppose the \(c_2\) scores are normal with positive variance and have observed mean \(1.0\) and sample standard deviation \(2.0\). Use familywise level \(0.05\).
For \(c_1'\mu\), the exact equality of occasions implies that the target is zero, and the interval \([0,0]\) covers it with probability one. A nondegenerate \(t\) pivot cannot be formed by dividing by its zero sample standard deviation; in particular, it is incorrect to claim that this interval has failure probability \(0.025\).
For \(c_2'\mu\), use a two-sided interval with marginal noncoverage \(0.025\), namely \(1.0\pm t_{7,\,0.9875}(2.0/\sqrt8)\). The union bound gives familywise noncoverage at most \(0+0.025=0.025\), which is no greater than \(0.05\). This conservative guarantee remains valid even though the first interval’s actual failure probability is zero. In a less deterministic setting, a zero-variance contrast must be justified by the model or design; a sample standard deviation of zero alone does not establish that its population variance is zero.
Common Pitfalls and Interpretation
- Treating a marginal interval as a familywise interval. Using the ordinary \(1-\alpha\) interval separately for every contrast does not generally give \(1-\alpha\) coverage for all of them together.
- Assuming Bonferroni requires independent contrasts. It does not; its proof uses the union bound. Dependence may affect how conservative the result is, but it does not invalidate the bound.
- Using a \(t\) pivot when a contrast score has zero variance. The usual studentized statistic is undefined in that case. If the score is known to be constant at its target, treat its interval as degenerate with coverage one; otherwise do not claim the ordinary exact \(t\) result.
- Applying a multivariate region beyond its dimension or assumptions. The stated Hotelling region requires \(n>q\), a positive-definite contrast covariance, and multivariate normality. A singular contrast covariance or too many linearly independent contrasts prevents this stated construction.
- Defining the family after seeing the results. A guarantee applies only to the family actually covered by the procedure. Searching across additional occasions or contrasts and then reporting them as if they belonged to the original smaller family changes the inferential target.
Check Your Understanding
Use the family and model assumptions explicitly in each answer.
- Why does the Bonferroni union-bound argument not require independence among within-subject contrasts?
- For \(m\) positive-variance normal contrast scores, what marginal noncoverage probability should each two-sided Bonferroni interval use at familywise level \(\alpha\)?
- What conditions are needed for the stated Hotelling region for \(q\) contrast means?
- Why do projections of a joint confidence ellipsoid provide simultaneous rather than merely marginal intervals?
- A contrast score is identically zero under the design. Why is its failure probability not \(\alpha/m\), and how can it be included in a union-bound argument?
- How should the family be defined if all pairwise occasion differences were searched but only selected results will be reported?