Orthogonal Contrasts and Independent Gaussian Comparisons

Why Orthogonality Matters for Planned Comparisons

A planned comparison summarizes a particular difference or pattern among several group means. When two such comparisons are orthogonal, their estimates can carry separate Gaussian information: under a common-covariance Gaussian model, the estimates are independent. That result gives a useful way to organize planned comparisons, especially when a scientific question can be broken into distinct components.

The word orthogonal needs care. With unequal group sizes, ordinary perpendicularity of the weight vectors is not generally the right condition. The sampling variances of the group means determine the relevant geometry.

Model and Contrast Orthogonality

Suppose there are $G\geq 2$ independent groups. In group $g$, the observations $Y_{g1},\ldots,Y_{gn_g}$ are independent and identically distributed as $N_p(\mu_g,\Sigma)$, and observations from different groups are independent as well. Here $n_g\geq 1$, $\mu_g$ is a $p$-vector, and the common covariance matrix $\Sigma$ is positive definite. Write $\bar Y_g$ for the sample mean in group $g$.

Definition: A group contrast is a nonzero vector of weights $a=(a_1,\ldots,a_G)^T$ satisfying $\sum_{g=1}^G a_g=0$. Its estimated multivariate contrast is $\hat\delta_a=\sum_{g=1}^G a_g\bar Y_g$, with mean $\delta_a=\sum_{g=1}^G a_g\mu_g$. Two contrasts $a$ and $b$ are orthogonal for this allocation when $\sum_{g=1}^G a_gb_g/n_g=0$.

The zero-sum condition makes the contrast insensitive to adding the same vector to every group mean. The allocation-specific orthogonality condition accounts for the fact that $\operatorname{Cov}(\bar Y_g)=\Sigma/n_g$: a group with a smaller sample size contributes more sampling variance. In particular, when all group sizes are equal, this condition reduces to $a^Tb=0$.

Covariance and Independence

Theorem: Under the model above, for any two group contrasts $a$ and $b$, $$ E[\hat\delta_a]=\delta_a,\qquad \operatorname{Cov}(\hat\delta_a,\hat\delta_b) =\left(\sum_{g=1}^G\frac{a_gb_g}{n_g}\right)\Sigma. $$ If $a$ and $b$ are orthogonal for the allocation, then $\hat\delta_a$ and $\hat\delta_b$ are independent.

Proof. Linearity of expectation gives $E[\hat\delta_a]=\sum_g a_g\mu_g=\delta_a$. Group means from distinct groups are independent, so their cross-covariances are zero. Within group $g$, $\operatorname{Cov}(\bar Y_g)=\Sigma/n_g$. Therefore

\operatorname{Cov}(\hat\delta_a,\hat\delta_b) =\sum_{g=1}^G\sum_{h=1}^G a_gb_h\operatorname{Cov}(\bar Y_g,\bar Y_h) =\sum_{g=1}^G\frac{a_gb_g}{n_g}\Sigma.

The group means are jointly multivariate normal, so the two contrast estimates are jointly multivariate normal. If the contrasts are orthogonal, their cross-covariance is zero. To see why this yields independence, let $s,t\in\mathbb R^p$. The joint characteristic function of the two centered estimates at $(s,t)$ is the exponential of minus one half their joint covariance quadratic form. A zero cross-covariance makes that quadratic form the sum of the separate forms in $s$ and $t$. Thus the joint characteristic function factors into the two marginal characteristic functions, which proves independence. $\square$

The same argument applies to any finite collection of contrasts: if every pair is orthogonal for the allocation, their estimates are mutually independent. In particular, each nonzero contrast has covariance $\left(\sum_g a_g^2/n_g\right)\Sigma$, which is positive definite because $\Sigma$ is positive definite and the weights are not all zero.

Worked Example: Equal Group Sizes

Three groups each have $n_g=12$ observations, and $p=2$. Take $a=(1,-1,0)^T$ and $b=(1,1,-2)^T$. Both weight vectors sum to zero, so both define contrasts. Check orthogonality:

\sum_{g=1}^3\frac{a_gb_g}{n_g} =\frac{1(1)+(-1)(1)+0(-2)}{12}=0.

They are therefore orthogonal for this allocation. Their variance factors are

\sum_{g=1}^3\frac{a_g^2}{n_g}=\frac{1+1}{12}=\frac16, \qquad \sum_{g=1}^3\frac{b_g^2}{n_g}=\frac{1+1+4}{12}=\frac12.

For any positive-definite common covariance $\Sigma$, the estimates have covariance matrices $\operatorname{Cov}(\hat\delta_a)=\Sigma/6$ and $\operatorname{Cov}(\hat\delta_b)=\Sigma/2$, and their cross-covariance is the zero matrix. Joint normality then makes the two estimates independent. The result concerns their sampling distributions, not whether two realized estimates look numerically unrelated.

Decomposing a Planned Contrast

Orthogonal contrasts provide coordinates for the space of all group contrasts. This makes it possible to express a planned comparison as a sum of independent comparison estimates. To describe the geometry, define the weighted inner product

\langle a,b\rangle_D=\sum_{g=1}^G\frac{a_gb_g}{n_g}, \qquad D=\operatorname{diag}(1/n_1,\ldots,1/n_G).

This is an inner product because every $n_g$ is positive: for nonzero $a$, $\langle a,a\rangle_D=\sum_g a_g^2/n_g>0$. The contrast vectors form the subspace $C=\{a\in\mathbb R^G:\sum_g a_g=0\}$, which has dimension $G-1$.

Result: There is a basis $a^{(1)},\ldots,a^{(G-1)}$ of the contrast space that is pairwise orthogonal under $\langle\cdot,\cdot\rangle_D$. For any contrast $c$, there are unique coefficients $\lambda_1,\ldots,\lambda_{G-1}$ such that $$ c=\sum_{k=1}^{G-1}\lambda_k a^{(k)} \quad\text{and hence}\quad \hat\delta_c=\sum_{k=1}^{G-1}\lambda_k\hat\delta_{a^{(k)}}. $$ The component estimates in this expression are independent, and $$ \operatorname{Cov}(\hat\delta_c) =\left(\sum_{k=1}^{G-1}\lambda_k^2 \langle a^{(k)},a^{(k)}\rangle_D\right)\Sigma. $$

Proof. Start with any basis of the $(G-1)$-dimensional subspace $C$. Apply the Gram–Schmidt construction using $\langle\cdot,\cdot\rangle_D$: retain the first basis vector, then from each next vector subtract its projections onto the vectors already constructed. Each resulting vector remains in $C$, since it is a linear combination of vectors in $C$. No resulting vector is zero: a zero residual would make the corresponding original basis vector a linear combination of earlier ones, contradicting linear independence. The resulting vectors are therefore a basis and are pairwise orthogonal. Since they are a basis, each contrast $c$ has a unique expansion as displayed. Linearity of the estimate gives the stated decomposition. The component estimates are independent by the theorem. Finally, expand the covariance of their weighted sum. All cross-covariances vanish, leaving the sum of the component covariance matrices shown above. $\square$

This decomposition concerns the estimated contrast vector and its covariance. It does not claim that the squared Mahalanobis length of a sum is the sum of the squared Mahalanobis lengths of its components.

Worked Example: Orthogonality With Unequal Group Sizes

Let $G=3$, with sample sizes $n_1=10$, $n_2=20$, and $n_3=30$. Consider $a=(1,-1,0)^T$ and $b=(1,2,-3)^T$; each sums to zero. Their ordinary dot product is $a^Tb=1-2=-1$, so they are not perpendicular in the unweighted geometry. But the allocation-specific calculation is

\langle a,b\rangle_D =\frac{1(1)}{10}+\frac{(-1)(2)}{20}+\frac{0(-3)}{30} =\frac{1}{10}-\frac{1}{10}=0.

Thus their estimates are independent under the stated Gaussian model, despite the nonzero ordinary dot product. Their variance factors are

\langle a,a\rangle_D=\frac{1}{10}+\frac{1}{20}=\frac{3}{20}, \qquad \langle b,b\rangle_D=\frac{1}{10}+\frac{4}{20}+\frac{9}{30} =\frac35.

Consequently, $\operatorname{Cov}(\hat\delta_a)=(3/20)\Sigma$, $\operatorname{Cov}(\hat\delta_b)=(3/5)\Sigma$, and $\operatorname{Cov}(\hat\delta_a,\hat\delta_b)=0$. The larger variance factor for $b$ follows directly from its weights and the group sample sizes.

Worked Example: Expressing a Planned Comparison in Independent Components

Use the unequal-size groups from the previous calculation and their orthogonal contrasts $a=(1,-1,0)^T$ and $b=(1,2,-3)^T$. Suppose the planned contrast is $c=(2,-1,-1)^T$. It sums to zero. To find its coordinates, solve $c=\lambda a+\nu b$. The first and third coordinates give $\lambda+\nu=2$ and $-3\nu=-1$, so $\nu=1/3$ and $\lambda=5/3$. The second coordinate checks: $-5/3+2/3=-1$. Hence

\hat\delta_c=\frac53\hat\delta_a+\frac13\hat\delta_b.

The two component estimates are independent. Their covariance factors add with squared coefficients:

\left(\frac53\right)^2\frac{3}{20} +\left(\frac13\right)^2\frac35 =\frac{5}{12}+\frac{1}{15} =\frac{29}{60}.

A direct calculation verifies the result:

\langle c,c\rangle_D =\frac{2^2}{10}+\frac{(-1)^2}{20}+\frac{(-1)^2}{30} =\frac{2}{5}+\frac{1}{20}+\frac{1}{30} =\frac{29}{60}.

Thus $\operatorname{Cov}(\hat\delta_c)=(29/60)\Sigma$. This representation separates the planned comparison into two independent Gaussian components without changing the contrast being estimated.

What Independence Does—and Does Not—Give

Independent contrast estimates can clarify which parts of a planned comparison contribute to its uncertainty. If the null mean pattern makes each component mean zero, the independent estimates also separate the corresponding Gaussian noise. This can be useful when organizing comparisons around scientifically meaningful components rather than treating all group differences as interchangeable.

There are two important limits. First, the independence result depends on the model: it uses independent groups, a common covariance matrix, and Gaussian observations. The covariance formula alone still shows zero cross-covariance under the common-covariance assumptions, but without joint normality zero covariance by itself does not establish independence. Second, independence of contrast estimates does not automatically imply independence of test statistics formed from them. In particular, statistics that use the same estimated covariance matrix can remain dependent because they share that random estimate.

Common Mistake: Using the Wrong Orthogonality Condition

A common error is to check $a^Tb=0$ for unequal group sizes and conclude that the estimates are independent. That condition ignores the different variances of the group means. In the unequal-size example, $a^Tb=-1$, while the correct weighted inner product is zero; the estimates are independent because their cross-covariance is zero under the model. Conversely, ordinary perpendicularity need not give independence when sample sizes differ. Always check $\sum_g a_gb_g/n_g=0$, then use joint normality to infer independence.

A second caution is not to include the zero vector as a contrast in an independence claim. Its estimate is deterministically zero, but it has no positive-definite covariance block. The definition here excludes it; any zero linear combination that arises in a decomposition is simply the constant zero vector.

Key takeaway: Under independent Gaussian groups with common positive-definite covariance, two nonzero group contrasts have independent estimates exactly when their weights are orthogonal in the allocation-weighted inner product $\sum_g a_gb_g/n_g$. An orthogonal basis then decomposes any planned contrast into independent estimated components.

Check Your Understanding

Use the common-covariance Gaussian model and the definitions in this tutorial.

For two groups of equal size, how does allocation-specific orthogonality relate to the ordinary dot product of the contrast weights?
For sample sizes $(8,16,24)$, are $a=(1,-1,0)^T$ and $b=(1,2,-3)^T$ orthogonal for the allocation? Show the weighted calculation.
Why does zero cross-covariance imply independence for the contrast estimates in this model, and why need it not do so without Gaussianity?
If a contrast is expressed as a linear combination of pairwise orthogonal contrasts, how is its covariance factor calculated?
Does independence of two contrast estimates guarantee independence of two test statistics that use the same estimated covariance matrix? Explain.