Simultaneous Inference for Robust Multivariate Contrasts

Why Simultaneous Inference Is Needed

A robust analysis may estimate several group locations and then compare many prespecified differences. Reporting an ordinary 95% confidence interval for each difference does not generally give 95% confidence that all intervals cover their targets. Even when every estimate is robust to outlying observations, the collection of separate inferences can have a higher probability of at least one error than the nominal level.

The goal here is to control uncertainty for a family of contrasts as a family. The contrasts may be dependent: for example, three pairwise differences among three group locations are linearly related. A joint ellipsoid uses that dependence and the effective dimension of the family. Bonferroni adjustment is a simpler alternative that controls familywise error without requiring the contrasts to be independent.

Contrast Families and Their Covariance

Suppose there are \(g\) groups, each with a \(d\)-dimensional location parameter \(\mu_i\). Stack the parameters into \(\theta=(\mu_1^\top,\ldots,\mu_g^\top)^\top\), a vector of length \(p=gd\). Let \(\hat\theta\) be a robust estimator of this vector. A prespecified family of scalar contrasts can be written as \(C\theta\), where \(C\) is an \(m\times p\) matrix. Its \(j\)th row specifies one contrast, and the target for that row is \(\delta_j=c_j^\top\theta\).

A vector-valued comparison, such as the difference between two \(d\)-dimensional group locations, can be included by using one row of \(C\) for each component. The family must be specified before looking at the estimated results if its stated multiplicity control is to apply to that family.

For the asymptotic procedures below, assume the robust estimator satisfies \(\sqrt{n}(\hat\theta-\theta)\xrightarrow{d}N(0,V)\), where \(V\) is finite and positive definite, and that an estimator \(\hat V\) satisfies \(\hat V\xrightarrow{p}V\). These are assumptions to be justified for the robust estimator and sampling design being used; they do not follow merely from calling an estimator robust. The estimated covariance of \(\hat\theta\) is then \(\hat V/n\), and the estimated covariance of \(C\hat\theta\) is \(C\hat V C^\top/n\).

If \(C\) has dependent rows, let \(q=\operatorname{rank}(C)\), and select a \(q\times p\) matrix \(D\) whose rows are a basis for the row space of \(C\). Then \(C=RD\) for some \(m\times q\) matrix \(R\). Write \(\eta=D\theta\), so the full family is \(C\theta=R\eta\). The covariance matrix \(D V D^\top\) is positive definite because the rows of \(D\) are linearly independent and \(V\) is positive definite. The number \(q\), not necessarily the number \(m\) of listed contrasts, is the effective dimension of the family.

Joint Ellipsoids and Simultaneous Bounds

A joint confidence ellipsoid is formed for the independent contrast coordinates \(\eta=D\theta\). Its projection onto any one contrast gives a simultaneous interval for that contrast. The dimension in the chi-square critical value is \(q\), so redundant rows of \(C\) do not inflate the critical value.

Proof. By the assumed limit and the definition of \(D\), \(\sqrt n(\hat\eta-\eta)\xrightarrow{d}N(0,\Sigma)\), where \(\Sigma=DVD^\top\). Covariance consistency gives \(\hat\Sigma\xrightarrow{p}\Sigma\). Since \(\Sigma\) is positive definite, matrix inversion is continuous at \(\Sigma\), and thus \(\hat\Sigma^{-1}\xrightarrow{p}\Sigma^{-1}\). Slutsky's theorem now gives

Therefore the probability that the statistic is no greater than \(\chi^2_{q,1-\alpha}\) tends to \(1-\alpha\), proving the ellipsoid's asymptotic coverage. On that ellipsoid, write \(h=\hat\eta-\eta\). For every row \(r_j^\top\), the Cauchy–Schwarz inequality in the \(\hat\Sigma^{-1}\) metric gives \(|r_j^\top h|\leq\sqrt{h^\top\hat\Sigma^{-1}h}\sqrt{r_j^\top\hat\Sigma r_j}\). The ellipsoid bounds the first factor by \(\sqrt{\chi^2_{q,1-\alpha}/n}\). Hence every \(\eta\) in the ellipsoid satisfies all the stated interval bounds simultaneously. The joint coverage of the intervals is therefore at least the coverage of the ellipsoid, asymptotically at least \(1-\alpha\). \(\square\)

The same construction gives a joint Wald test. To test \(H_0:C\theta=\delta_0\), first check that the proposed \(\delta_0\) is compatible with the row dependencies in \(C=RD\). If it is, express it as \(R\eta_0\) and test \(H_0:\eta=\eta_0\) with \(W=n(\hat\eta-\eta_0)^\top\hat\Sigma^{-1}(\hat\eta-\eta_0)\). Reject at asymptotic level \(\alpha\) when \(W>\chi^2_{q,1-\alpha}\). Under the null, the same limit argument proves that \(W\xrightarrow{d}\chi^2_q\).

Adjusted Critical Values: Bonferroni Inference

A direct alternative is to construct a separate interval for each of the \(m\) contrasts, but use a more stringent critical value. For asymptotically normal scalar contrasts with consistent standard errors, a two-sided Bonferroni interval for contrast \(j\) uses the critical value \(z_{1-\alpha/(2m)}\), where \(z_u\) is the \(u\) quantile of a standard normal distribution. This approach does not require estimating the correlations among contrasts.

Justification. Let \(A_j\) be the event that interval \(j\) fails to cover \(\delta_j\). The union bound gives \(P(\bigcup_{j=1}^m A_j)\leq\sum_{j=1}^m P(A_j)\). If each summand has limit superior at most \(\alpha/m\), then the limit superior of the familywise noncoverage probability is at most \(m(\alpha/m)=\alpha\). No independence assumption is used. This bound can be conservative, especially when the contrasts are strongly correlated.

Choosing and Interpreting a Procedure

Joint ellipsoidal bounds use the estimated covariance among contrasts and can be substantially narrower than Bonferroni bounds when contrasts are dependent. Their critical value reflects the rank \(q\) of the contrast family. Bonferroni bounds are straightforward and remain valid under arbitrary dependence, but their critical value depends on the number \(m\) of reported contrasts, including redundant ones. A procedure based on the maximum of correlated standardized contrasts can also use their joint dependence, but its critical value must be calibrated for the specified joint distribution; it is not automatically the ordinary one-contrast normal cutoff.

The robust estimator and the simultaneous procedure solve different problems. Robust location estimation and its covariance estimate address sensitivity to nonnormality or contamination under their assumptions. Simultaneous inference addresses the chance of making one or more errors across a family of questions. Robustness alone does not correct multiplicity, and multiplicity adjustment does not make a nonrobust estimator resistant to contamination.

Common Pitfalls

• Calling separate 95% intervals a simultaneous 95% procedure. Each interval may have 95% marginal coverage, while the chance that all cover is lower. Use a joint procedure or adjust the critical values.
• Using the number of listed contrasts as the ellipsoid dimension. The correct chi-square degrees of freedom are the rank \(q\). Three pairwise differences among three scalar locations, for example, have rank two.
• Dividing an estimator covariance by \(n\) twice. If a covariance estimate is already for \(\hat\delta\), use it directly. Divide an estimate of the asymptotic covariance of \(\sqrt n(\hat\delta-\delta)\) by \(n\) only when that is the scale being reported.
• Choosing the family after inspecting the results. The stated simultaneous guarantee applies to the prespecified family, not to an unaccounted-for selection of contrasts.