Asymptotic Simultaneous Intervals From Joint Normality

Why Joint Normality Leads to Simultaneous Intervals

A collection of individually valid confidence intervals does not automatically provide the desired coverage for the whole collection. For simultaneous inference, the relevant event is that every interval covers its target at once. When a vector of estimators is jointly asymptotically normal, this joint event can be calibrated through the limiting distribution of the largest standardized error across the contrasts.

The method applies to a fixed, prespecified, finite family of contrasts. Its calibration uses both the marginal scales and the dependence among the contrast estimates. A consistent covariance estimate gives the estimated interval widths; the joint limiting distribution determines the common critical value.

Definition: Let $\hat\theta$ estimate a $p$-dimensional parameter $\theta$, and let $C$ be a fixed $m\times p$ matrix whose rows $c_1^\top,\ldots,c_m^\top$ specify the contrasts. The contrast targets are $c_j^\top\theta$, for $j=1,\ldots,m$. A set of intervals is simultaneous at level $1-\alpha$ when the probability that all $m$ intervals cover their respective targets is $1-\alpha$, or approaches $1-\alpha$ asymptotically.

The Limiting Maximum and Its Critical Value

Suppose that $p$ and $m$ are fixed and, as $n$ increases,

\sqrt{n}(\hat\theta-\theta)\xrightarrow{d}N(0,V),

where $V$ is a fixed positive semidefinite covariance matrix. Assume that each contrast has positive limiting variance: $c_j^\top Vc_j>0$ for every $j$. Define the correlation matrix $R$ of the limiting contrast errors by

R_{jk}= \frac{c_j^\top Vc_k} {\sqrt{(c_j^\top Vc_j)(c_k^\top Vc_k)}}.

Let $Z=(Z_1,\ldots,Z_m)^\top$ have a mean-zero multivariate normal distribution with covariance matrix $R$, and define $M=\max_{1\leq j\leq m}|Z_j|$. Each $Z_j$ has variance one, though the components need not be independent. For $0<\alpha<1$, choose $q_{1-\alpha}$ so that

P(M\leq q_{1-\alpha})=1-\alpha.

The distribution of $M$ is continuous: if $M=x$, then at least one component satisfies $|Z_j|=x$, and each such event has probability zero because $Z_j$ is a nondegenerate normal variable. Thus a critical value with the stated probability can be used. It is a joint critical value, not generally the usual marginal normal critical value.

Theorem: Suppose $\sqrt{n}(\hat\theta-\theta)\xrightarrow{d}N(0,V)$, $C$ and $m$ are fixed, and $c_j^\top Vc_j>0$ for all $j$. Let $\hat V\xrightarrow{p}V$, and assume the estimated contrast variances $c_j^\top\hat Vc_j$ are positive with probability tending to one. Define $$ T_n=\max_{1\leq j\leq m} \frac{\sqrt{n}\,|c_j^\top(\hat\theta-\theta)|} {\sqrt{c_j^\top\hat Vc_j}}. $$ Then $T_n\xrightarrow{d}M$, where $M$ is the maximum absolute component of a mean-zero multivariate normal vector with correlation matrix $R$.

Proof. Write $Y_n=\sqrt{n}(\hat\theta-\theta)$. The contrast vector $CY_n$ converges in distribution to $CY$, where $Y\sim N(0,V)$, by the continuous mapping theorem. For each $j$, consistency of $\hat V$ gives $\sqrt{c_j^\top\hat Vc_j}\xrightarrow{p}\sqrt{c_j^\top Vc_j}$. The limiting denominator is positive by assumption. Because there are finitely many contrasts, the vector of all standardized coordinates therefore converges jointly, by Slutsky’s theorem, to the vector with components \[ \frac{c_j^\top Y}{\sqrt{c_j^\top Vc_j}}. \] This vector is mean-zero normal and has correlation matrix $R$. Taking the maximum of the absolute coordinates is a continuous operation, so the continuous mapping theorem gives $T_n\xrightarrow{d}M$. $\square$

Intervals and Their Coverage

Using the same critical value for every contrast gives the intervals

c_j^\top\hat\theta \ \pm\ q_{1-\alpha}\sqrt{\frac{c_j^\top\hat Vc_j}{n}}, \qquad j=1,\ldots,m.

The estimated standard error in the $j$th interval is $\sqrt{c_j^\top\hat Vc_j/n}$. The use of a common critical value makes the simultaneous event exactly the event that the maximum standardized error is no larger than that value.

Theorem: Under the assumptions of the preceding theorem, if $q_{1-\alpha}$ satisfies $P(M\leq q_{1-\alpha})=1-\alpha$, the intervals displayed above have asymptotic simultaneous coverage $1-\alpha$: $$ P\!\left( c_j^\top\theta\text{ is in its interval for every }j \right)\longrightarrow 1-\alpha. $$

Proof. The event that every interval covers its target is equivalent to \[ \max_j \frac{\sqrt{n}\,|c_j^\top(\hat\theta-\theta)|} {\sqrt{c_j^\top\hat Vc_j}} \leq q_{1-\alpha}, \] which is the event $T_n\leq q_{1-\alpha}$. The preceding theorem gives $T_n\xrightarrow{d}M$. Since the distribution of $M$ is continuous at $q_{1-\alpha}$, convergence in distribution implies \[ P(T_n\leq q_{1-\alpha})\longrightarrow P(M\leq q_{1-\alpha})=1-\alpha. \] This proves the claimed simultaneous coverage. $\square$

The limiting correlation matrix is part of the calibration: changing the dependence among contrasts can change the distribution of their maximum. If $R$ is not known and a critical value is computed from an estimated correlation matrix, consistency of that estimate alone is not the entire argument. One also needs continuity of the relevant quantile as a function of the correlation matrix at the true value. Under conditions ensuring that continuity, the estimated critical value converges to $q_{1-\alpha}$, and the same coverage argument applies. The examples below use a known limiting correlation structure to keep the calibration explicit.

Worked Example: Two Independent Limiting Contrast Errors

Suppose $m=2$, the two standardized limiting contrast errors are independent standard normal variables, and $\alpha=0.05$. For $M=\max(|Z_1|,|Z_2|)$, independence gives \[ P(M\leq q)=\{2\Phi(q)-1\}^2. \] Setting this equal to $0.95$ yields \[ 2\Phi(q)-1=\sqrt{0.95},\qquad \Phi(q)=\frac{1+\sqrt{0.95}}{2}\approx0.98734, \] so $q\approx2.24$.

For a numerical illustration, take $n=100$, $\hat\theta=(1.2,-0.4)^\top$, $C$ equal to the $2\times2$ identity matrix, and $\hat V=\operatorname{diag}(1,4)$. The estimated standard errors are \[ \sqrt{1/100}=0.10,\qquad \sqrt{4/100}=0.20. \] The intervals are therefore \[ 1.2\pm2.24(0.10)=[0.976,1.424], \qquad -0.4\pm2.24(0.20)=[-0.848,0.048], \] to three decimal places. Their joint coverage approaches $0.95$ under the stated asymptotic model.

Worked Example: Perfectly Correlated Standardized Errors

Let $\theta=(\theta_1,\theta_2)^\top$, $V=I_2$, and consider $c_1=(1,0)^\top$ and $c_2=(2,0)^\top$. The two targets are $\theta_1$ and $2\theta_1$. Although they have different scales, their standardized limiting errors are identical: \[ \frac{c_1^\top Y}{\sqrt{c_1^\top Vc_1}}=Y_1, \qquad \frac{c_2^\top Y}{\sqrt{c_2^\top Vc_2}}=\frac{2Y_1}{2}=Y_1. \] Thus $M=|Y_1|$, and at level $0.95$ the joint critical value is the ordinary two-sided normal value $q\approx1.96$. The second contrast adds no new standardized error event.

For example, let $n=64$, $\hat\theta_1=3.1$, and $\hat V=\operatorname{diag}(1.21,0.81)$. The estimated standard errors for the two contrasts are \[ \sqrt{1.21/64}=1.1/8=0.1375, \qquad \sqrt{4(1.21)/64}=2.2/8=0.275. \] The intervals are $3.1\pm1.96(0.1375)=[2.8305,3.3695]$ and $6.2\pm1.96(0.275)=[5.661,6.739]$. Their simultaneous calibration follows from the single underlying standardized error, not from treating the two contrasts as independent.

Worked Example: Three Independent Contrasts With Unequal Standard Errors

Suppose three standardized limiting errors are independent standard normal variables and $\alpha=0.05$. Then \[ P(M\leq q)=\{2\Phi(q)-1\}^3, \qquad \Phi(q)=\frac{1+0.95^{1/3}}{2}\approx0.9915. \] This gives $q\approx2.39$. Now take $n=144$, $C=I_3$, $\hat\theta=(0.8,1.5,-0.2)^\top$, and $\hat V=\operatorname{diag}(0.25,1,2.25)$. The three standard errors are \[ \sqrt{0.25/144}=\frac{0.5}{12}\approx0.0417,\quad \sqrt{1/144}=\frac{1}{12}\approx0.0833,\quad \sqrt{2.25/144}=\frac{1.5}{12}=0.125. \]

Multiplying these by $2.39$ gives half-widths approximately $0.0996$, $0.1992$, and $0.2988$. The simultaneous intervals are consequently $[0.7004,0.8996]$, $[1.3008,1.6992]$, and $[-0.4988,0.0988]$. Unequal estimated variances change the widths, while the shared maximum-based critical value accounts for requiring all three intervals to cover together.

Common Mistake: Reusing a Marginal Critical Value

A frequent error is to use the ordinary two-sided $1-\alpha$ normal critical value in every interval and conclude that the whole family has coverage $1-\alpha$. That critical value controls each marginal interval, not the probability that all intervals cover simultaneously. For instance, with two independent standard normal errors, using $1.96$ gives simultaneous coverage \[ P(|Z_1|\leq1.96,\ |Z_2|\leq1.96)=0.95^2=0.9025, \] not $0.95$. The correction is to calibrate the maximum $M$ at the desired familywise level, using the joint limiting distribution. Its critical value depends on the dependence structure; marginal normality alone does not determine it.

The result is asymptotic rather than an exact finite-sample guarantee. It also assumes a fixed contrast family and a covariance estimate consistent for the covariance in the estimator’s limiting distribution. In particular, the covariance must be on the correct scale: if $\sqrt n(\hat\theta-\theta)$ has limiting covariance $V$, then the contrast variance estimate used in the interval is $c_j^\top\hat Vc_j/n$, not $c_j^\top\hat Vc_j$.

Key takeaway: For a fixed family of contrasts, form each estimated standard error from the consistent covariance estimate, then use a critical value from the limiting distribution of the largest absolute standardized contrast error. This targets simultaneous, rather than merely marginal, asymptotic coverage.

Check Your Understanding

Check that you can connect the estimator’s joint limit to the coverage of the entire interval family.

Why must every contrast have positive limiting variance for the standardized maximum in this construction?
What determines the correlation matrix of the limiting standardized contrast errors?
Explain why the event that all intervals cover is equivalent to an inequality involving $T_n$.
For two independent standard normal errors, write the equation that defines the $1-\alpha$ critical value of their absolute maximum.
What additional condition may be needed when the maximum critical value is computed from an estimated correlation matrix?

Asymptotic Simultaneous Intervals From Joint Normality

What You'll Learn