Simultaneous Coverage and Test-Confidence Duality

From a Joint Region to Simultaneous Inference

When several contrasts are reported together, the relevant question is not only whether each interval is individually plausible. We also want a guarantee that the entire collection covers its corresponding true values with a specified probability. An exact joint confidence region provides one route: project it onto each contrast to obtain intervals, then use those same intervals to define tests.

The projection step has an important consequence. If the joint region contains the true contrast vector, every projected interval contains its true contrast. The reverse need not hold: several separate projection constraints can include points that lie outside the joint region. Thus the resulting intervals have simultaneous coverage at least as large as the joint region's coverage, but their coverage need not be exactly equal to it.

Model and notation: Suppose $Y\sim N(X\beta,\sigma^2 I_n)$, where $X$ is a fixed, full-column-rank $n\times p$ design matrix, $\beta\in\mathbb R^p$, and $\nu=n-p>0$. Let $\hat\beta=(X'X)^{-1}X'Y$, and let $s^2=\operatorname{RSS}/\nu$ be the usual residual mean square. A contrast $c'\beta$ is estimable when $c$ belongs to the row space of $X$.

Consider a nonempty finite family of nonzero estimable contrasts $c_1'\beta,\ldots,c_m'\beta$. Their coefficient vectors may be linearly dependent. Let their span have dimension $r$, and choose an $r\times p$ matrix $B$ whose rows form a basis for that span. Write $\theta=B\beta$, $\hat\theta=B\hat\beta$, and $A=B(X'X)^{-1}B'$. Because the rows of $B$ are independent and estimable, $A$ is positive definite. Every contrast in the family can be written $c_j'\beta=a_j'\theta$ for some $a_j\in\mathbb R^r$. Since each $c_j$ is nonzero, $a_j\ne0$; positive definiteness of $A$ then gives $c_j'(X'X)^{-1}c_j=a_j'Aa_j>0$.

Definition: For $0<\alpha<1$, the exact joint $1-\alpha$ confidence region for $\theta$ is $$ \mathcal E_{1-\alpha} = \left\{\theta_0: (\hat\theta-\theta_0)'A^{-1}(\hat\theta-\theta_0) \leq r s^2 F_{r,\nu}(1-\alpha) \right\}, $$ where $F_{r,\nu}(1-\alpha)$ is the $1-\alpha$ quantile of the central $F$ distribution with $r$ and $\nu$ degrees of freedom.

Why the Joint Region Has Exact Coverage

The region is calibrated by an exact $F$ pivot. The estimator $\hat\theta$ is normal with mean $\theta$ and covariance matrix $\sigma^2 A$, so the standardized quadratic form has a chi-square distribution with $r$ degrees of freedom. In the normal linear model, the residual sum of squares is independent of $\hat\beta$, and $\nu s^2/\sigma^2$ has a chi-square distribution with $\nu$ degrees of freedom. Their ratio gives the stated $F$ law.

Theorem: Under the fixed-design normal model above, for every $\beta$ and every $\sigma^2>0$, $$ P\left(\theta\in\mathcal E_{1-\alpha}\right)=1-\alpha. $$

Proof. Under the model, $\hat\theta-\theta\sim N(0,\sigma^2 A)$. Since $A$ is positive definite, $$ \frac{(\hat\theta-\theta)'A^{-1}(\hat\theta-\theta)}{\sigma^2} \sim \chi^2_r. $$ Also, $\nu s^2/\sigma^2\sim\chi^2_\nu$, independently of $\hat\theta$. Therefore $$ \frac{(\hat\theta-\theta)'A^{-1}(\hat\theta-\theta)}{r s^2} = \frac{\{(\hat\theta-\theta)'A^{-1}(\hat\theta-\theta)/\sigma^2\}/r} {\{\nu s^2/\sigma^2\}/\nu} \sim F_{r,\nu}. $$ The event $\theta\in\mathcal E_{1-\alpha}$ is exactly the event that this statistic is no greater than $F_{r,\nu}(1-\alpha)$. A continuous $F_{r,\nu}$ distribution assigns probability $1-\alpha$ to that event. $\square$

Projecting the Region to Get Intervals

For contrast $a_j'\theta=c_j'\beta$, the projection of the ellipsoid in that direction is an interval centered at $a_j'\hat\theta$. Its half-width depends on the variance factor $a_j'Aa_j=c_j'(X'X)^{-1}c_j$. The same joint critical value is used for every contrast in the family.

Theorem: A simultaneous interval for each contrast $c_j'\beta$, $j=1,\ldots,m$, is $$ a_j'\hat\theta \ \pm\ \sqrt{rF_{r,\nu}(1-\alpha)\,s^2\,a_j'Aa_j} \;=\; c_j'\hat\beta \ \pm\ \sqrt{rF_{r,\nu}(1-\alpha)\,s^2\,c_j'(X'X)^{-1}c_j}. $$ The probability that all these intervals cover their respective true contrasts is at least $1-\alpha$.

Proof. If $\theta\in\mathcal E_{1-\alpha}$, put $d=\hat\theta-\theta$. For any $a_j$, the Cauchy–Schwarz inequality gives $$ |a_j'd| = |(A^{1/2}a_j)'(A^{-1/2}d)| \leq \sqrt{a_j'Aa_j}\sqrt{d'A^{-1}d}. $$ Inside the ellipsoid, $d'A^{-1}d\leq r s^2F_{r,\nu}(1-\alpha)$. Substitution gives $$ |a_j'\hat\theta-a_j'\theta| \leq \sqrt{rF_{r,\nu}(1-\alpha)\,s^2\,a_j'Aa_j}. $$ This holds for every $j$, so joint-region coverage implies coverage by all the intervals. Consequently, $$ P(\text{all intervals cover}) \geq P(\theta\in\mathcal E_{1-\alpha}) =1-\alpha. $$ This proves the simultaneous coverage guarantee. $\square$

Worked Example: Projecting a Two-Contrast Ellipsoid

Suppose $r=2$, $\nu=9$, $s^2=4$, and the covariance factor for the two basis contrasts is $A=\operatorname{diag}(1/4,1/9)$. Suppose their estimates are $10$ and $-3$. At the 95% level, $F_{2,9}(0.95)\approx4.2565$, so $rF_{r,\nu}(0.95)\approx 8.513$.

For the first contrast, the half-width is $$ \sqrt{8.513\cdot4\cdot\frac14}=\sqrt{8.513}\approx2.918. $$ Its interval is therefore approximately $10\pm2.918$, or $(7.082,12.918)$.

For the second contrast, the half-width is $$ \sqrt{8.513\cdot4\cdot\frac19} =\sqrt{3.7836}\approx1.945. $$ Its interval is approximately $-3\pm1.945$, or $(-4.945,-1.055)$. Together these intervals cover both true contrasts with probability at least $0.95$. The guarantee applies to the pair as a family, not merely to each interval considered separately.

Inverting the Intervals Gives Familywise-Valid Tests

For each contrast, consider the two-sided null hypothesis $H_{0j}:c_j'\beta=\theta_{0j}$. The interval test rejects $H_{0j}$ exactly when the null value $\theta_{0j}$ lies outside its simultaneous interval. Equivalently, it rejects when $$ \frac{|c_j'\hat\beta-\theta_{0j}|} {s\sqrt{c_j'(X'X)^{-1}c_j}} > \sqrt{rF_{r,\nu}(1-\alpha)}. $$ This is a common critical value for all members of the specified family. The tests need not have the same individual rejection probability, but the family has a strong familywise error guarantee.

Theorem: The interval-inversion tests control the familywise error rate at level $\alpha$ under every configuration of true and false null hypotheses. That is, the probability of rejecting at least one true null in the family is at most $\alpha$.

Proof. Let $T$ be the set of indices for which $H_{0j}$ is true. If a true null is rejected, its null value equals the true contrast but lies outside its interval. Thus at least one true contrast is not covered. The event of at least one false rejection is therefore contained in the event that not all family intervals cover their true contrasts. By the simultaneous coverage result, $$ P(\text{at least one true null is rejected}) \leq P(\text{not all family intervals cover}) \leq \alpha. $$ This argument does not require every null in the family to be true; it applies to any subset $T$, which is why the control is strong. $\square$

Worked Example: Testing Two Contrasts by Interval Inversion

Suppose a family has $r=2$ contrasts and $\nu=10$ residual degrees of freedom. At $\alpha=0.05$, $F_{2,10}(0.95)\approx4.1028$, so the simultaneous standardized critical value is $$ \sqrt{2F_{2,10}(0.95)} =\sqrt{8.2056}\approx2.8645. $$ Assume each contrast has estimated standard error $2$, and the estimates are $6$ and $5$.

For the first contrast, the standardized distance from the null value $0$ is $6/2=3$, which exceeds $2.8645$. Its simultaneous interval has half-width $2(2.8645)=5.729$, giving approximately $(0.271,11.729)$. Since $0$ is outside, the test rejects $H_{01}:c_1'\beta=0$.

For the second contrast, the standardized distance is $5/2=2.5$, which does not exceed $2.8645$. Its interval is approximately $5\pm5.729=(-0.729,10.729)$, which contains $0$, so the test does not reject $H_{02}:c_2'\beta=0$. The procedure's familywise guarantee applies to the two decisions together, including when only one of these null hypotheses is true.

Why Simultaneous Coverage Can Exceed the Joint Coverage

The intervals are projections, so their joint coverage event is generally larger than the event that the full vector lies in the ellipsoid. For a concrete geometric check, consider a two-dimensional ellipsoid centered at the origin with $A=I_2$ and squared radius $c>0$. Its points satisfy $u_1^2+u_2^2\leq c$. The two coordinate projections instead require $|u_1|\leq\sqrt c$ and $|u_2|\leq\sqrt c$.

Worked Example: Spanning Contrasts Do Not Reproduce the Ellipsoid

The two coordinate directions span $\mathbb R^2$. Nevertheless, take $u=(0.8\sqrt c,0.8\sqrt c)$. Both coordinate constraints hold, because each absolute coordinate is $0.8\sqrt c\leq\sqrt c$. But $$ u_1^2+u_2^2 =0.64c+0.64c =1.28c >c. $$ Thus both projected intervals can cover while the vector lies outside the joint ellipsoid. Spanning the contrast space alone does not make the simultaneous interval event equivalent to ellipsoid coverage.

For a nonsingular normal distribution, the region between the ellipse and its enclosing projection rectangle has positive probability in this two-dimensional example. Hence the simultaneous coverage can be strictly greater than $1-\alpha$, not just geometrically larger as a set. Equality requires the collection of projection constraints to characterize the ellipsoid (up to probability-zero boundaries); a finite family of directions in dimension greater than one generally does not do so.

Common Mistake and Practical Interpretation

Wrong reasoning: “The contrasts span the parameter space, so covering every listed contrast is equivalent to covering the joint ellipsoid.”
Correction: Spanning says the contrasts contain enough directions to identify the vector, but their interval constraints need not impose the ellipsoid's full quadratic bound. The projected intervals inherit a lower bound on simultaneous coverage, not automatic equality.
Another pitfall: Treating each interval as an ordinary single-contrast interval while claiming familywise control. The stated guarantee comes from the joint critical value $rF_{r,\nu}(1-\alpha)$, not from using an unadjusted one-contrast critical value for every test.

The practical benefit of the construction is a direct connection between estimation and testing. Report the simultaneous intervals when the goal is to communicate plausible values for a family of contrasts; reject a family member's null value precisely when it falls outside its interval. The same coverage argument supports both interpretations and protects against false rejections across the specified family.

Key takeaway: An exact joint confidence ellipsoid projects to simultaneous intervals with coverage at least $1-\alpha$. Inverting those intervals yields tests with strong familywise error control at level $\alpha$. Projection may enlarge the coverage event, so equality with the ellipsoid's coverage is not guaranteed.

Check Your Understanding

Use the definitions and arguments above to answer the following questions.

Why is the dimension $r$, rather than the number $m$ of listed contrasts, used in the joint $F$ critical value?
State the event-containment argument that gives simultaneous interval coverage of at least $1-\alpha$.
How does excluding a null value from its simultaneous interval define a two-sided test?
Why does the familywise error proof apply when only some of the null hypotheses are true?
Explain why two contrast directions that span a plane need not make their projected intervals equivalent to a joint ellipsoid.