Simultaneous Comparisons With a Reference Coordinate

Why a Shared Reference Changes the Comparison Family

In many studies, one coordinate has a special role: a control group, a baseline measurement, or a reference condition. The scientific questions are then not all possible pairwise differences, but whether each of the other means differs from this same reference. These comparisons form a focused family, but they are dependent because they share the reference coordinate.

That dependence matters for simultaneous inference. A procedure that controls the error probability for each interval separately does not automatically control the probability of at least one incorrect interval. We will define the family precisely, construct intervals with guaranteed familywise coverage when the covariance is unknown, and see how the dependence can be used for a sharper calibration when the covariance is known.

Definition: Let $X_1,\ldots,X_n$ be independent observations from a $p$-variate normal distribution $N_p(\mu,\Sigma)$, where $\mu=(\mu_1,\ldots,\mu_p)'$. Choose a reference coordinate $r\in\{1,\ldots,p\}$. For every $j\ne r$, define the reference contrast $c_j=e_j-e_r$ and its mean contrast $\theta_j=c_j'\mu=\mu_j-\mu_r$. The family contains the $m=p-1$ contrasts $\theta_j$, one for each nonreference coordinate.

Here $e_j$ is the $j$th coordinate vector. The direction of subtraction is part of the definition: a positive $\theta_j$ means coordinate $j$ has a larger mean than the reference. Reversing every contrast would reverse the signs of the estimates and interval endpoints, without changing their simultaneous coverage.

How the Contrasts Depend on One Another

Write $\bar X$ for the sample mean vector and $S$ for the sample covariance matrix, with divisor $n-1$. The estimated contrast is $c_j'\bar X=\bar X_j-\bar X_r$. Since the sample mean has covariance matrix $\Sigma/n$, the covariance between two estimated contrasts is determined by the two contrast vectors:

\operatorname{Cov}(c_j'\bar X,c_k'\bar X) =\frac{c_j'\Sigma c_k}{n} =\frac{\Sigma_{jk}-\Sigma_{jr}-\Sigma_{rk}+\Sigma_{rr}}{n}.

Even if the coordinates themselves are mutually uncorrelated, the contrasts are not independent: both contain the same reference observation with a minus sign. This is a structural feature of the comparison family, not a consequence of a particular data set.

Result: If $\Sigma=I_p$, then each reference contrast has variance $2/n$, and any two distinct reference contrasts have covariance $1/n$. Their correlation is therefore $1/2$.

To verify the result, take distinct $j,k\ne r$. Under $\Sigma=I_p$, the only nonzero terms in $c_j'\Sigma c_k$ are the product of the two reference-coordinate coefficients, $(-1)(-1)=1$. Also, $c_j'\Sigma c_j=c_j'c_j=2$. Dividing the covariance $1/n$ by the two standard deviations $\sqrt{2/n}$ gives correlation $1/2$. With a general covariance matrix, the correlations need not equal $1/2$, or even be positive; the displayed covariance formula gives the relevant values.

Worked Example: Defining the Reference Family

Suppose $p=4$, and coordinate 2 is the reference. The family has three, not six, comparisons:

\theta_1=\mu_1-\mu_2,\qquad \theta_3=\mu_3-\mu_2,\qquad \theta_4=\mu_4-\mu_2.

Their contrast vectors are $e_1-e_2$, $e_3-e_2$, and $e_4-e_2$. The comparison $\mu_1-\mu_3$, for example, is not one of the specified reference comparisons. It could be obtained as $\theta_1-\theta_3$, but that does not make it part of this family or give it a simultaneous interval automatically.

Unknown Covariance: Finite-Sample Bonferroni Intervals

When $\Sigma$ is unknown, an easily implemented finite-sample procedure is Bonferroni calibration. For each contrast define its estimated standard error by $s_j=\sqrt{c_j'Sc_j/n}$. Under the multivariate normal model, if $c_j'\Sigma c_j>0$, the statistic $(c_j'\bar X-\theta_j)/s_j$ has a Student $t$ distribution with $n-1$ degrees of freedom. This is a marginal statement for each contrast; it does not assert independence across the family.

Theorem: Suppose $X_1,\ldots,X_n$ are independent and identically distributed as $N_p(\mu,\Sigma)$, $n\ge2$, and $c_j'\Sigma c_j>0$ for every $j\ne r$. Let $t_{n-1,\,1-\alpha/(2m)}$ denote the $1-\alpha/(2m)$ quantile of the $t_{n-1}$ distribution. Then the intervals $$ c_j'\bar X\ \pm\ t_{n-1,\,1-\alpha/(2m)}\sqrt{\frac{c_j'Sc_j}{n}}, \qquad j\ne r, $$ cover all $m$ reference contrasts simultaneously with probability at least $1-\alpha$.

Proof. Let $A_j$ be the event that interval $j$ fails to cover $\theta_j$. The marginal $t$ result and the two-sided quantile imply $P(A_j)=\alpha/m$. The probability of at least one failure is bounded by the union bound:

P\left(\bigcup_{j\ne r}A_j\right) \le \sum_{j\ne r}P(A_j) =m\frac{\alpha}{m} =\alpha.

Taking complements gives probability at least $1-\alpha$ that every interval covers its contrast. This proof requires no independence among the failure events, so it remains valid despite the shared reference. $\square$

Worked Example: Bonferroni Intervals With an Estimated Covariance

Suppose $p=3$, coordinate 3 is the reference, $n=10$, and the observed sample mean and sample covariance are

\bar X=(5,4,3.5)',\qquad S= \begin{pmatrix} 4&1&0.5\\ 1&9&1\\ 0.5&1&1 \end{pmatrix}.

For a 95% simultaneous family, $m=2$, $\alpha=0.05$, and the Bonferroni critical value is $t_{9,0.9875}\approx2.685$. The two estimated contrasts are $5-3.5=1.5$ and $4-3.5=0.5$. Their estimated variances before division by $n$ are

(e_1-e_3)'S(e_1-e_3)=4+1-2(0.5)=4,\qquad (e_2-e_3)'S(e_2-e_3)=9+1-2(1)=8.

Thus their standard errors are $\sqrt{4/10}\approx0.63246$ and $\sqrt{8/10}\approx0.89443$. The half-widths, using the displayed critical value, are $2.685(0.63246)\approx1.698$ and $2.685(0.89443)\approx2.401$. The resulting intervals are approximately $(-0.198,3.198)$ for $\mu_1-\mu_3$ and $(-1.901,2.901)$ for $\mu_2-\mu_3$. Together they have at least 95% coverage under the stated model.

The estimated covariance of the two sample contrasts is $(1-0.5-1+1)/10=0.05$. Their estimated correlation is $0.5/\sqrt{4\cdot8}\approx0.088$. This particular covariance gives a modest observed correlation; it does not alter the validity of the Bonferroni guarantee.

Known Covariance: Calibrating the Joint Maximum

If $\Sigma$ is known, the standardized errors of the sample contrasts have a joint multivariate normal distribution with a known correlation matrix. In that setting, one can calibrate the maximum absolute error directly rather than allocate an error probability separately to each interval. Define $Z_j=(c_j'\bar X-\theta_j)/\sqrt{c_j'\Sigma c_j/n}$, and let $R$ be the correlation matrix of the vector $Z$. Choose $q_\alpha$ so that $P(\max_{j\ne r}|Z_j|\le q_\alpha)=1-\alpha$. Then the intervals with half-width $q_\alpha\sqrt{c_j'\Sigma c_j/n}$ have exactly $1-\alpha$ simultaneous coverage, provided the quantile is chosen from this joint distribution.

This exact calibration uses the dependence, because $R$ is the correlation matrix induced by the reference contrasts. It is not the same as applying a univariate normal critical value to each interval. For unknown $\Sigma$, replacing it by an estimate and treating the resulting critical value as exact is not justified by this known-covariance argument.

Worked Example: Shared-Reference Dependence Changes the Critical Value

Let $p=3$, use coordinate 3 as reference, suppose $\Sigma=I_3$ is known, and take $n=4$. The two contrasts are $\mu_1-\mu_3$ and $\mu_2-\mu_3$. By the covariance calculation above, their standardized errors are standard normal with correlation $1/2$. The exact 95% critical value $q_{0.05}$ is determined by

P(|Z_1|\le q_{0.05},|Z_2|\le q_{0.05})=0.95, \qquad (Z_1,Z_2)'\sim N_2\left((0,0)',\begin{pmatrix}1&1/2\\1/2&1\end{pmatrix}\right).

Numerical evaluation of this bivariate normal probability gives a critical value of about $2.22$ (approximately $2.22$ to two decimal places). If independence were incorrectly assumed, the 95% Šidák critical value would be $\Phi^{-1}((1+\sqrt{0.95})/2)\approx2.236$. Bonferroni gives $\Phi^{-1}(1-0.05/4)\approx2.241$. These are different calibrations: the true standardized errors share a positive correlation, and the joint probability must be evaluated using that correlation rather than an independence formula.

For illustration, suppose the observed sample mean is $(2,1,0.5)'$. Each contrast estimate has standard error $\sqrt{2/4}=0.7071$, because each contrast has variance 2 under $\Sigma=I_3$. Using the joint critical value rounded to $2.22$, the common half-width is $2.22(0.7071)\approx1.570$. The simultaneous intervals are approximately $(-0.070,3.070)$ for the first contrast and $(-1.070,2.070)$ for the second. The critical value is shown rounded, so these endpoints are illustrative; coverage is determined by the unrounded quantile from the joint probability equation.

Common Mistakes and Practical Interpretation

Using separate 95% intervals as a 95% family. Each marginal interval may cover with probability 0.95, while the probability that all intervals cover is smaller. Familywise calibration addresses the joint event.
Assuming reference contrasts are independent. Even independent coordinates produce correlated differences when each one subtracts the same reference. Compute the contrast covariance rather than guessing it.
Applying Šidák without its independence condition. The Šidák formula uses independence of the relevant error events. Shared-reference contrasts generally violate that condition; use a valid Bonferroni bound or the correct joint calibration.
Adding every pairwise difference to the family. A reference family has $p-1$ members. All $\binom p2$ pairwise differences are a different family and need their own calibration.

The choice of reference should be scientifically meaningful and specified as part of the comparison question. A simultaneous interval containing zero does not establish equality; it indicates that the procedure has not ruled out zero at the chosen familywise level. The interval width also reflects both the variability of the coordinates and their covariance with the reference.

Key takeaway: Reference comparisons are contrasts $e_j-e_r$, and their shared reference induces dependence. Bonferroni intervals control familywise coverage without an independence assumption; when the covariance is known, the joint maximum can use the actual dependence for exact calibration.

Check Your Understanding

Use the definitions and results above to answer the following.

For $p=5$ coordinates and reference coordinate $r=4$, how many contrasts are in the reference family? Write one of them explicitly.
If $\Sigma=I_p$, derive the variance and correlation of two distinct sample-mean reference contrasts.
Why does the Bonferroni proof remain valid when reference contrasts are dependent?
What assumption makes the joint normal maximum calibration in the final worked example exact?
Explain why a 95% interval for each contrast separately does not generally imply 95% simultaneous coverage for the whole family.