Worked One-Way Analysis With Planned Group Contrasts

Purpose and Model

A one-way multivariate analysis compares several group mean vectors using a shared estimate of within-group variation. When the comparisons of interest are specified in advance, the analysis can focus on those planned contrasts rather than treating every possible comparison as equally important. The key is to account both for the covariance among response variables and for the fact that several inferences are being made.

Consider \(g\) independent groups, each measured on \(p\) outcomes. Group \(i\) has \(n_i\) observations \(X_{ij}\), with mean vector \(\mu_i\). The common-covariance Gaussian model is

X_{ij}\sim N_p(\mu_i,\Sigma),\qquad i=1,\ldots,g,\quad j=1,\ldots,n_i,

where observations are independent across groups and individuals, and \(\Sigma\) is a common positive-definite \(p\times p\) covariance matrix. Write \(N=\sum_i n_i\) and \(\nu=N-g\). The group sample means are \(\bar X_i\), and the within-group sum-of-products-and-cross-products matrix is

W=\sum_{i=1}^{g}\sum_{j=1}^{n_i}(X_{ij}-\bar X_i)(X_{ij}-\bar X_i)',\qquad S_p=\frac{W}{\nu}.

The matrix \(S_p\) pools the within-group covariance information. It is the common covariance estimate used for inference here. A planned contrast is a linear combination of the group means with coefficients that sum to zero.

Definition: For fixed coefficients \(a_1,\ldots,a_g\) satisfying \(\sum_i a_i=0\), the vector contrast is \(c=\sum_i a_i\mu_i\). Its estimator is \(\hat c=\sum_i a_i\bar X_i\), and its variance scale is \(q=\sum_i a_i^2/n_i\). Under the common-covariance model, \(\operatorname{Cov}(\hat c)=q\Sigma\), estimated by \(qS_p\).

Why the Pooled Estimate and Contrast Scale Work

The pooled matrix and the factor \(q\) have distinct roles. The pooled matrix estimates the covariance of a single observation within a group; \(q\) adjusts that estimate for the particular weighted combination of group means and their sample sizes.

Result: Under the stated model, \(W\) has a Wishart distribution with scale \(\Sigma\) and \(\nu\) degrees of freedom, so \(S_p\) is unbiased for \(\Sigma\). Also, \(\hat c\sim N_p(c,q\Sigma)\), and \(\hat c\) is independent of \(S_p\).

To see the contrast covariance, independence of groups gives \(\operatorname{Cov}(\bar X_i)=\Sigma/n_i\) and zero cross-covariances between distinct group means. Therefore

\operatorname{Cov}(\hat c) =\operatorname{Cov}\left(\sum_i a_i\bar X_i\right) =\sum_i a_i^2\frac{\Sigma}{n_i} =\left(\sum_i\frac{a_i^2}{n_i}\right)\Sigma=q\Sigma.

The distribution and independence statements follow from the standard Gaussian sampling results for group means and within-group scatter: within each group, the sample mean is independent of its residual sum-of-products matrix, and the latter is Wishart with \(n_i-1\) degrees of freedom. Independent Wishart matrices with the same scale add their degrees of freedom, giving \(\nu=\sum_i(n_i-1)=N-g\). These facts explain why a common covariance assumption is essential to pooling.

Exact Test for One Planned Vector Contrast

For the null hypothesis \(H_0:c=c_0\), use the squared Mahalanobis distance between the estimated and hypothesized contrast, measured with its estimated covariance:

T^2=(\hat c-c_0)'(qS_p)^{-1}(\hat c-c_0).

When \(\nu\ge p\), under the common-covariance Gaussian model and \(H_0\), the exact Hotelling transformation is

\frac{\nu-p+1}{p\nu}T^2\sim F_{p,\nu-p+1}.

In fact, \((\hat c-c_0)/\sqrt q\) is a \(N_p(0,\Sigma)\) vector independent of \(S_p\), and \(W=\nu S_p\) is Wishart with \(\nu\) degrees of freedom. The displayed result is the standard Hotelling \(T^2\) distribution for an independent normal vector and Wishart covariance estimate. Thus the upper-tail probability from this \(F\) distribution is an exact p-value, not an approximation.

Worked Example: Estimating a Contrast From Three Groups

Suppose three groups each have \(n_i=4\) observations on two outcomes. Their sample means are \(\bar X_1=(10,20)'\), \(\bar X_2=(12,21)'\), and \(\bar X_3=(15,19)'\). Suppose each group's within-group sum-of-products matrix is \(6I_2\). These summaries are feasible, for example with centered residuals \(\sqrt3 e_1,-\sqrt3 e_1,\sqrt3 e_2,-\sqrt3 e_2\) in each group. Compare group 3 with group 1 using \(a=(-1,0,1)\).

The contrast estimate and its variance scale are

\hat c=-\bar X_1+\bar X_3=(5,-1)',\qquad q=\frac{(-1)^2}{4}+\frac{0^2}{4}+\frac{1^2}{4}=\frac12.

The total within-group matrix is \(W=18I_2\), and \(\nu=12-3=9\). Therefore \(S_p=W/\nu=2I_2\), and the estimated contrast covariance is \(qS_p=I_2\). To test \(H_0:c=0\),

T^2=(5,-1)I_2^{-1}(5,-1)'=25+1=26, \qquad F=\frac{9-2+1}{2(9)}\,26=\frac{104}{9}\approx11.5556.

The reference distribution is \(F_{2,8}\). For \(F_{2,8}\), the upper-tail probability at \(f\) is \(\{8/(8+2f)\}^4\). Hence

p=\left(\frac{8}{8+2(104/9)}\right)^4 =\left(\frac{9}{35}\right)^4 =\frac{6561}{1500625}\approx0.00437.

This is evidence against a zero difference in the two-dimensional mean vector. The test assesses the vector jointly; it does not separately test each outcome at the same significance level.

Several Prespecified Contrasts and Familywise Error

Suppose \(m\) vector contrasts are selected before examining the data. Each can be tested with its own exact Hotelling p-value. Even if every individual test has level \(\alpha\), using all \(m\) tests without adjustment can make the probability of at least one false rejection exceed \(\alpha\). The Bonferroni method controls this familywise error without requiring independence among the tests.

Result: If \(p_k\) is a valid p-value for the \(k\)th null hypothesis, then the adjusted p-value is \(p_{k,\mathrm{adj}}=\min(1,mp_k)\). Rejecting when \(p_{k,\mathrm{adj}}\le\alpha\) controls the probability of one or more false rejections across the family at no more than \(\alpha\).

For the proof, let \(I_0\) be the set of true null hypotheses. For any true null, validity gives \(P(p_k\le\alpha/m)\le\alpha/m\). The probability of at least one false rejection is bounded by the union bound:

P\left(\bigcup_{k\in I_0}\{p_k\le\alpha/m\}\right) \le \sum_{k\in I_0}P(p_k\le\alpha/m) \le |I_0|\frac{\alpha}{m} \le\alpha.

No independence assumption is used. The price of this robustness is that Bonferroni can be conservative, especially when many comparisons are strongly related.

Worked Example: Bonferroni-Adjusted Planned Comparisons

Use the three-group summaries above and test two prespecified contrasts: \(c_1=\mu_3-\mu_1\), and \(c_2=\mu_2-(\mu_1+\mu_3)/2\). The first test was worked out above, giving \(p_1\approx0.00437\). For the second contrast the coefficients are \((-1/2,1,-1/2)\), which sum to zero.

Its estimate and variance scale are

\hat c_2=(12,21)'-\frac12\{(10,20)'+(15,19)'\} =(-0.5,1.5)',\qquad q_2=\frac{1/4+1+1/4}{4}=\frac38.

Since \(S_p=2I_2\), the estimated covariance is \(q_2S_p=(3/4)I_2\). Thus

T_2^2=(-0.5,1.5)\left(\frac34 I_2\right)^{-1}(-0.5,1.5)' =\frac{0.25+2.25}{0.75}=\frac{10}{3}, \qquad F_2=\frac{8}{18}\frac{10}{3}=\frac{40}{27}\approx1.4815.

Using the \(F_{2,8}\) upper tail gives

p_2=\left(\frac{8}{8+2(40/27)}\right)^4 =\left(\frac{27}{37}\right)^4\approx0.2836.

There are \(m=2\) planned tests. The adjusted p-values are

p_{1,\mathrm{adj}}=\min(1,2p_1)\approx0.00874,\qquad p_{2,\mathrm{adj}}=\min(1,2p_2)\approx0.5672.

At familywise level \(0.05\), reject the first null but not the second. The conclusion is limited to this prespecified family; it does not imply that every possible contrast has been tested.

Simultaneous Confidence Regions

A Bonferroni family of tests can also be expressed through simultaneous confidence regions. For contrast \(k\), let \(q_k=\sum_i a_{ki}^2/n_i\), and let \(F_{p,\nu-p+1;1-\alpha/m}\) denote the \(1-\alpha/m\) quantile of the corresponding \(F\) distribution. The region is

(\hat c_k-c_k)'(q_kS_p)^{-1}(\hat c_k-c_k) \le \frac{p\nu}{\nu-p+1}F_{p,\nu-p+1;1-\alpha/m}.

For each contrast, this region has coverage at least \(1-\alpha/m\). By the same union-bound argument, the probability that all \(m\) regions cover their respective true contrasts is at least \(1-\alpha\). This is a simultaneous coverage guarantee for the family, even though the regions need not be independent.

Worked Example: A Simultaneous Region for a Vector Difference

For the two contrasts above, take \(\alpha=0.05\), so the quantile uses \(1-\alpha/m=0.975\). Here \(p=2\), \(\nu=9\), and for the first contrast \(q_1S_p=I_2\). The \(F_{2,8}\) upper-tail formula gives the quantile explicitly by solving \(\{8/(8+2f)\}^4=0.025\):

f=\frac{4}{0.025^{1/4}}-4\approx6.06,\qquad \frac{p\nu}{\nu-p+1}f=\frac{18}{8}f\approx13.63.

The simultaneous region for \(c_1\) is therefore the disk

(c_1-(5,-1)')'(c_1-(5,-1)')\le13.63.

Its radius is approximately \(\sqrt{13.63}=3.69\). The origin is at distance \(\sqrt{5^2+(-1)^2}=\sqrt{26}\approx5.10\) from the center, so it is outside the region. This agrees with rejecting the first null using the Bonferroni-adjusted test. The region describes uncertainty in the entire contrast vector, preserving the dependence between its two outcome coordinates.

Common Pitfalls and Practical Interpretation

Forgetting the contrast variance scale. The covariance estimate is \(qS_p\), not just \(S_p\). Omitting \(q\) changes the test statistic, and the error depends on both the coefficients and group sample sizes.
Inverting the wrong matrix or miscomputing a quadratic form. For example, \((4I_2)^{-1}=I_2/4\); a vector \((1,-1)'\) then has quadratic form \(2/4=1/2\), not \(8\). Check the inverse and each coordinate contribution before calculating an \(F\) statistic or p-value.
Using unadjusted p-values for a family. Each exact test may be valid on its own, but a collection of tests needs a stated familywise procedure if the overall error rate is to be controlled.
Pooling under covariance heterogeneity. The exact reference distribution here depends on a common \(\Sigma\). If group covariance matrices differ, this pooled-covariance analysis is not justified; unequal-covariance methods are needed.
Changing the planned family after seeing results. The multiplicity guarantee pertains to the family actually declared. Selecting contrasts based on observed group differences and then treating them as prespecified does not preserve the same guarantee.

The practical sequence is direct: verify the common-covariance model, calculate the pooled within-group estimate, define each contrast and its \(q\) factor, obtain exact Hotelling p-values, and adjust the declared family. Reporting the coefficient vectors, sample sizes, pooled covariance, and adjusted results makes the analysis reproducible.

Check Your Understanding

Work through these questions using the definitions and results above.

For a contrast with coefficients \((1,-1,0)\) and group sizes \((5,8,6)\), calculate \(q\).
Why does the exact Hotelling test require the contrast estimate to be independent of the pooled covariance estimate?
If four valid planned-comparison p-values are \(0.01, 0.03, 0.20,\) and \(0.60\), calculate their Bonferroni-adjusted values.
Explain why the Bonferroni familywise guarantee does not require the planned tests to be independent.
What model assumption fails if the group covariance matrices are different, and which part of the pooled analysis then loses its justification?