Max-t Critical Values for Simultaneous Inference

Why Calibrate the Maximum Statistic?

For a family of contrasts, separate confidence intervals can each have the desired marginal coverage while failing to cover all target values together at that level. Simultaneous intervals address the family as a whole. A max-t procedure does this by finding the distribution of the largest absolute, standardized estimation error across the specified contrasts, then choosing a critical value from that distribution.

The critical value depends on the joint distribution of the contrast statistics. In particular, the statistics are generally dependent, even when their numerators are uncorrelated: in a normal-theory model they share the same estimate of the error variance. Accounting for that joint structure is what distinguishes max-t calibration from applying a single-contrast t critical value repeatedly.

Model and Max-t Definition

Suppose $Y=X\beta+\varepsilon$, where $X$ is a fixed, full-column-rank design matrix, and the errors are independent $N(0,\sigma^2)$. Let $p$ be the number of columns of $X$, $n$ the number of observations, and $\nu=n-p>0$. Write $\hat\beta=(X^\top X)^{-1}X^\top Y$ and $s^2=\mathrm{SSE}/\nu$. Consider a prespecified family of $m$ contrasts $a_j^\top\beta$, with $a_j\ne 0$.

Definition: The estimated $j$th contrast is $\hat\theta_j=a_j^\top\hat\beta$, and its estimated standard error is $s\sqrt{a_j^\top(X^\top X)^{-1}a_j}$. Its standardized estimation error is $$ T_j=\frac{\hat\theta_j-a_j^\top\beta}{s\sqrt{a_j^\top(X^\top X)^{-1}a_j}}. $$ The max-t statistic for this family is $M=\max_{1\le j\le m}|T_j|$. A level-$1-\alpha$ max-t critical value $c_\alpha$ is chosen so that $P(M\le c_\alpha)=1-\alpha$.

The contrast statistics have a joint multivariate t distribution with $\nu$ degrees of freedom. More precisely, define $$ R_{jk}=\frac{a_j^\top(X^\top X)^{-1}a_k}{\sqrt{a_j^\top(X^\top X)^{-1}a_j}\sqrt{a_k^\top(X^\top X)^{-1}a_k}}. $$ Then their joint distribution has the representation $$ (T_1,\ldots,T_m)^\top=\frac{Z}{\sqrt{U/\nu}}, $$ where $Z\sim N_m(0,R)$, $U\sim\chi^2_\nu$, and $Z$ and $U$ are independent. The matrix $R$ is a correlation matrix and may be singular, for example if some contrasts are proportional. The same random denominator appears in every coordinate. Consequently, zero correlation between two coordinates does not in general make their t statistics independent.

Definition: For the specified contrast family and degrees of freedom, the max-t critical value is the $1-\alpha$ quantile of $M=\max_j|T_j|$ under the joint multivariate t distribution just described. Equivalently, it is the solution of $P(M\le c_\alpha)=1-\alpha$. This calibration depends on $R$, $\nu$, and the family size and structure.

From the Joint Quantile to Simultaneous Intervals

Once $c_\alpha$ is calibrated, use it for every contrast in the family. The same critical value is used throughout because the target is one joint coverage event, not $m$ unrelated coverage events.

Theorem: Under the normal linear model conditions stated above, if $c_\alpha$ satisfies $P(M\le c_\alpha)=1-\alpha$, then the intervals $$ \hat\theta_j\ \pm\ c_\alpha s\sqrt{a_j^\top(X^\top X)^{-1}a_j},\qquad j=1,\ldots,m, $$ have simultaneous coverage $1-\alpha$ for the specified contrast family.

Proof. For each $j$, the event that the $j$th interval contains its target is equivalent to $$ \left|a_j^\top\hat\beta-a_j^\top\beta\right| \le c_\alpha s\sqrt{a_j^\top(X^\top X)^{-1}a_j}, $$ which, after dividing by the positive standard-error factor, is exactly $|T_j|\le c_\alpha$. All intervals cover their targets if and only if these inequalities hold for every $j$, or equivalently if $M\le c_\alpha$. Therefore the simultaneous coverage probability is $P(M\le c_\alpha)=1-\alpha$. $\square$

For nonsingular or singular $R$, the distribution of $M$ has no point masses: $M=c$ implies that at least one coordinate satisfies $T_j=c$ or $T_j=-c$, each an event of probability zero. The distribution function is continuous. Thus its quantile can be calibrated to the requested coverage level; numerical methods can evaluate the joint probability when no closed form is available.

Worked Calibrations

Worked Example: Three Redundant Contrasts

Suppose three contrasts are $a_1^\top\beta=\theta$, $a_2^\top\beta=2\theta$, and $a_3^\top\beta=-\theta$. Let $\nu=10$, and suppose their estimates and estimated standard errors are, respectively, $1.5$ and $0.5$, $3.0$ and $1.0$, and $-1.5$ and $0.5$. Find the max-t critical value for 95% simultaneous coverage and the resulting intervals.

The three standardized errors are the same up to sign. For example, the second numerator and standard error are both twice those for the first contrast, so $T_2=T_1$; the third contrast reverses the sign, so $T_3=-T_1$. Hence $M=\max(|T_1|,|T_1|,|-T_1|)=|T_1|$, which has a univariate $t_{10}$ distribution. Therefore $c_{0.05}=t_{10,0.975}\approx2.228$.

The first interval is $1.5\pm2.228(0.5)=1.5\pm1.114$, or approximately $[0.386,2.614]$. The second is $3.0\pm2.228(1.0)$, or $[0.772,5.228]$. The third is $-1.5\pm2.228(0.5)$, or $[-2.614,-0.386]$. Since the simultaneous event reduces to $|T_1|\le2.228$, these three intervals cover their three targets together with probability $0.95$, not $0.95$ raised to the third power.

Worked Example: Calibrating Two Uncorrelated Contrasts with Two Residual Degrees of Freedom

Suppose two contrasts have correlation matrix $R=I_2$, $\nu=2$, and the desired simultaneous coverage is $0.95$. Their joint t distribution is not a pair of independent $t_2$ variables: it has independent normal numerators but a shared chi-square denominator. Use that representation to specify the exact critical value. Then suppose the estimated contrasts are $0.7$ and $-1.1$, with estimated standard errors $0.2$ and $0.3$.

Write $U\sim\chi^2_2$. Conditional on $U=u$, the two coordinates are independent normal variables with variance $2/u$. Therefore $$ P(M\le c)=\int_0^\infty \left[2\Phi\left(c\sqrt{u/2}\right)-1\right]^2\frac{1}{2}e^{-u/2}\,du. $$ With the change of variable $u=2t^2$, this becomes $$ P(M\le c)=\int_0^\infty 2t e^{-t^2}\left[2\Phi(ct)-1\right]^2\,dt. $$ Thus the exact max-t critical value is the unique positive solution $c_{0.05}$ of $$ \int_0^\infty 2t e^{-t^2}\left[2\Phi(c_{0.05}t)-1\right]^2\,dt=0.95. $$ This one-dimensional equation can be solved by numerical integration and root-finding; it specifies the critical value without treating the t coordinates as independent.

Using that critical value, the first interval is $0.7\pm0.2c_{0.05}$, and the second is $-1.1\pm0.3c_{0.05}$. Each interval uses its own standard error, but both use the same jointly calibrated critical value. By construction, the probability that both intervals cover is $0.95$.

Worked Example: The Known-Variance Normal Benchmark

As a benchmark, suppose two standardized contrast errors are independent standard normal variables because $\sigma$ is known, and the requested simultaneous coverage is $0.95$. Find the critical value, then construct intervals for estimates $2.0$ and $-0.5$ with standard errors $0.4$ and $0.6$.

For independent standard normal errors, $P(M\le c)=[P(|Z|\le c)]^2=[2\Phi(c)-1]^2$. Setting this equal to $0.95$ gives $2\Phi(c)-1=\sqrt{0.95}$, so $$ c=\Phi^{-1}\left(\frac{1+\sqrt{0.95}}{2}\right)\approx2.236. $$ The intervals are $2.0\pm2.236(0.4)=2.0\pm0.8944$, or approximately $[1.106,2.894]$; and $-0.5\pm2.236(0.6)=-0.5\pm1.3416$, or approximately $[-1.842,0.842]$. Their joint coverage is $0.95$. This is a normal, known-variance benchmark; with an estimated common variance, the finite-$\nu$ multivariate t calibration is needed instead.

Bonferroni as a Conservative Bound

A max-t critical value uses the joint distribution and can therefore exploit the dependence structure. A simpler alternative is to choose a critical value that guarantees the desired familywise coverage regardless of that structure.

Theorem: Let $t_{\nu,1-\alpha/(2m)}$ denote the $1-\alpha/(2m)$ quantile of a univariate $t_\nu$ distribution. Then $$ P\left(\max_{1\le j\le m}|T_j|\le t_{\nu,1-\alpha/(2m)}\right)\ge1-\alpha. $$ Consequently, the max-t critical value is no larger than this Bonferroni critical value.

Proof. Each $T_j$ has a marginal $t_\nu$ distribution, so $$ P\left(|T_j|>t_{\nu,1-\alpha/(2m)}\right)=\frac{\alpha}{m}. $$ The event that the maximum exceeds this critical value is the union of the $m$ events $\{|T_j|>t_{\nu,1-\alpha/(2m)}\}$. By the union bound, its probability is at most the sum of their probabilities, which is $m(\alpha/m)=\alpha$. Thus the probability that the maximum is at most the Bonferroni critical value is at least $1-\alpha$. Since the max-t critical value is the quantile attaining that coverage, it cannot be larger. $\square$

Calibration in Practice and a Common Pitfall

For a specified family, calibration requires the degrees of freedom and the correlation matrix $R$, both of which determine the joint t distribution of the standardized errors. When the maximum distribution has no convenient closed form, numerical integration or simulation from $Z/\sqrt{U/\nu}$ can be used. In either case, preserve the shared denominator in the simulation or integration: generate one $U$ for the whole vector, not a new independent denominator for each contrast.

Wrong reasoning: “Each contrast has a $t_\nu$ distribution, so the probability that all $m$ statistics lie between $-c$ and $c$ is the product of their marginal probabilities.”
Correction: Marginal distributions alone do not establish independence. In the normal-theory model the contrast statistics share $U$, and their dependence also reflects $R$. Use the joint multivariate t distribution to calibrate $M$, or use a bound such as Bonferroni when joint calibration is unavailable.

A second practical point is that the family must be specified before the calibration is used. Changing the contrasts changes $R$, and adding or removing contrasts can change the maximum distribution. A max-t critical value is not a universal multiplier for every possible collection of intervals.

Key takeaway: A max-t procedure calibrates one critical value to the distribution of the largest absolute standardized error in the chosen contrast family. Its simultaneous intervals have exact normal-theory coverage when that joint distribution and its degrees of freedom are used correctly.

Check Your Understanding

Try these questions using the definitions and results above.

Why can two contrast t statistics be dependent even when their normal numerators have correlation zero?
For a family of three contrasts, write the event that all max-t intervals cover their targets in terms of $M$.
If two contrasts are positive multiples of one another, what happens to their standardized errors and to the maximum absolute statistic?
Why does the Bonferroni argument not require the contrast statistics to be independent?
What information, beyond the degrees of freedom, is needed to calibrate a max-t critical value for a given contrast family?