Parameterizing Several Independent Group Means

Group Means as a Multivariate Parameter

In a one-way multivariate design, each independent group has a mean vector describing its average response across $p$ outcomes. The central task here is to write hypotheses about several such vectors in a way that makes their components and relationships explicit. This parameterization applies whether group covariance matrices are equal or unequal; it concerns the means, not a covariance assumption.

Suppose there are $g$ groups. In group $k$, let $Y_{ki}$ be the $p$-dimensional response vector for observation $i$, where $k=1,\ldots,g$ and $i=1,\ldots,n_k$. Assume the observations are independent across groups and within each group, and that each group has a finite mean vector $\mu_k=E[Y_{ki}]$. Write $\Sigma_k=\operatorname{Cov}(Y_{ki})$, which may differ across groups. The group mean vectors are the parameters of interest.

Definition: Stack the $g$ group mean vectors in group order to form the $gp$-dimensional parameter $$ \theta = \begin{pmatrix} \mu_1\\ \mu_2\\ \vdots\\ \mu_g \end{pmatrix}, \qquad \mu_k\in\mathbb{R}^p. $$ A linear hypothesis about these means has the form $A\theta=b$, where $A$ is a specified matrix and $b$ is a specified vector. A group contrast is a weighted sum of group mean vectors whose scalar weights sum to zero.

The group-ordering convention matters: the $p$ entries of $\mu_1$ come first in $\theta$, then the $p$ entries of $\mu_2$, and so on. For a $q\times g$ coefficient matrix $C$, the corresponding $q$ vector contrasts can be written together as

(C\otimes I_p)\theta,

where $I_p$ is the $p\times p$ identity matrix and $\otimes$ denotes the Kronecker product. The result is a $qp$-vector: it stacks the $q$ contrast vectors, each of length $p$. In particular, a row $c'=(c_1,\ldots,c_g)$ of $C$ produces the vector $\sum_{k=1}^g c_k\mu_k$.

Equality of All Group Mean Vectors

The equality hypothesis says that every group has the same mean for every outcome. One convenient way to encode it is to compare each of the first $g-1$ groups with group $g$. Define

C_{\mathrm{eq}}= \begin{pmatrix} 1&0&\cdots&0&-1\\ 0&1&\cdots&0&-1\\ \vdots&\vdots&\ddots&\vdots&\vdots\\ 0&0&\cdots&1&-1 \end{pmatrix}.

Then equality is the linear hypothesis $(C_{\mathrm{eq}}\otimes I_p)\theta=0$. Each block of $p$ equations says that one group mean vector equals the reference mean vector $\mu_g$.

Result: The hypothesis $(C_{\mathrm{eq}}\otimes I_p)\theta=0$ holds if and only if $\mu_1=\mu_2=\cdots=\mu_g$. The matrix $C_{\mathrm{eq}}$ has rank $g-1$, so the hypothesis supplies $(g-1)p$ independent scalar equations.

To verify the equivalence, suppose first that $(C_{\mathrm{eq}}\otimes I_p)\theta=0$. Its $i$th $p$-vector block is $\mu_i-\mu_g$, so each block being zero gives $\mu_i=\mu_g$ for $i=1,\ldots,g-1$. Thus all the group mean vectors are equal. Conversely, if all the vectors are equal, every difference $\mu_i-\mu_g$ is zero, so the displayed matrix equation holds. The first $g-1$ rows of $C_{\mathrm{eq}}$ are linearly independent: in any zero linear combination, the coefficient in column $i$, for $i<g$, is exactly the coefficient on row $i$, which must therefore be zero. Hence its rank is $g-1$. Because $I_p$ has rank $p$, $C_{\mathrm{eq}}\otimes I_p$ has rank $(g-1)p$. $\square$

Worked Example: Writing the Equality Hypothesis for Three Groups

Suppose each of three independent groups has two outcomes, and their sample mean vectors are $\bar Y_1=(4,10)'$, $\bar Y_2=(5,7)'$, and $\bar Y_3=(3,8)'$. First write the population equality hypothesis in the stacked parameterization, then evaluate the corresponding differences using these sample means.

The population parameter is $\theta=(\mu_1',\mu_2',\mu_3')'$, a six-vector. The equality matrix is

C_{\mathrm{eq}}\otimes I_2 = \begin{pmatrix} 1&0&-1&0&0&0\\ 0&1&0&-1&0&0\\ 1&0&0&0&-1&0\\ 0&1&0&0&0&-1 \end{pmatrix}.

Thus the null hypothesis is that this matrix times $\theta$ equals the four-vector of zeros. The two blocks compare groups 1 and 2, respectively, with group 3. Substituting the sample mean vectors into the same difference expressions gives

\begin{pmatrix} \bar Y_1-\bar Y_3\\ \bar Y_2-\bar Y_3 \end{pmatrix} = \begin{pmatrix} 4-3\\ 10-8\\ 5-3\\ 7-8 \end{pmatrix} = \begin{pmatrix} 1\\2\\2\\-1 \end{pmatrix}.

These are calculated sample differences, not the population hypothesis itself. Their nonzero values do not, by themselves, establish whether the population mean vectors are equal.

Planned Comparisons as Contrast Equations

A particular scientific question may concern a weighted combination of groups rather than equality of every group mean. For a coefficient vector $c=(c_1,\ldots,c_g)'$, the associated vector contrast is $\sum_k c_k\mu_k$. It is a contrast in the usual sense when $\sum_k c_k=0$. A planned hypothesis can set that vector equal to zero or to a prespecified target vector $r\in\mathbb{R}^p$:

(c'\otimes I_p)\theta=r.

The target need not be zero. For example, a study may ask whether the average of two treatment-group mean vectors exceeds a control mean vector by a specified amount. Several planned comparisons can be assembled by putting their coefficient vectors in the rows of $C$, giving $(C\otimes I_p)\theta=r$, with $r$ stacking the target vectors in the same order.

The zero-sum condition ensures that a contrast concerns differences among groups rather than an overall common shift. To see this precisely, add the same vector $a\in\mathbb{R}^p$ to every group mean. The contrast changes by

\sum_{k=1}^g c_k(\mu_k+a)-\sum_{k=1}^g c_k\mu_k = \left(\sum_{k=1}^g c_k\right)a.

If the weights sum to zero, this change is zero. Conversely, if the contrast is unchanged for every $a$, then $(\sum_k c_k)a=0$ for every vector $a$, which requires $\sum_k c_k=0$. This proves that zero-sum weights are exactly what makes the weighted sum invariant to a common shift in all group means. $\square$

Worked Example: Comparing the Average of Two Treatments With Control

A three-group study measures two outcomes. The groups are a low-dose treatment, a high-dose treatment, and control. Their population mean vectors are $\mu_1,\mu_2,\mu_3$. The planned comparison is the average of the two treatment means minus the control mean, with target $r=(2,-1)'$.

The weights are $c=(1/2,1/2,-1)'$, and they sum to $1/2+1/2-1=0$. The planned hypothesis is

\left( \begin{pmatrix}1/2&1/2&-1\end{pmatrix} \otimes I_2 \right)\theta = \begin{pmatrix}2\\-1\end{pmatrix}.

Suppose the mean vectors under consideration are $\mu_1=(12,4)'$, $\mu_2=(16,6)'$, and $\mu_3=(10,8)'$. The left-hand side is

\frac12\begin{pmatrix}12\\4\end{pmatrix} + \frac12\begin{pmatrix}16\\6\end{pmatrix} - \begin{pmatrix}10\\8\end{pmatrix} = \begin{pmatrix}6\\2\end{pmatrix} + \begin{pmatrix}8\\3\end{pmatrix} - \begin{pmatrix}10\\8\end{pmatrix} = \begin{pmatrix}4\\-3\end{pmatrix}.

For these vectors, the contrast differs from the target by $(4,-3)'-(2,-1)'=(2,-2)'$, so the stated equality does not hold at these values. The calculation describes the hypothesis in both outcome dimensions; it is not a statistical decision based on sample data.

Changing Coordinates Without Changing the Mean Questions

The stacked parameter $\theta$ is direct, but it is not the only useful coordinate system. For example, take group $g$ as a reference and define $\eta_0=\mu_g$ and $\eta_i=\mu_i-\mu_g$ for $i=1,\ldots,g-1$. The reference vector and the $g-1$ differences together determine all group means: $\mu_g=\eta_0$ and $\mu_i=\eta_0+\eta_i$. Thus this is an invertible reparameterization, not a restriction on the possible means. Equality of all group means becomes $\eta_1=\cdots=\eta_{g-1}=0$.

Worked Example: Rewriting a Contrast Using a Reference Group

For three groups with two outcomes, use group 3 as the reference and define $\eta_1=\mu_1-\mu_3$, $\eta_2=\mu_2-\mu_3$, and $\eta_0=\mu_3$. Consider the contrast $c=(1/2,1/2,-1)'$.

Since $c_1+c_2+c_3=0$, substitute $\mu_1=\eta_0+\eta_1$, $\mu_2=\eta_0+\eta_2$, and $\mu_3=\eta_0$:

\frac12\mu_1+\frac12\mu_2-\mu_3 = \frac12(\eta_0+\eta_1)+\frac12(\eta_0+\eta_2)-\eta_0 = \frac12\eta_1+\frac12\eta_2.

The common reference vector cancels, leaving a weighted combination of the two reference-group differences. For example, if $\mu_1=(12,4)'$, $\mu_2=(16,6)'$, and $\mu_3=(10,8)'$, then $\eta_1=(2,-4)'$ and $\eta_2=(6,-2)'$. The reparameterized contrast is $\tfrac12(2,-4)'+\tfrac12(6,-2)'=(4,-3)'$, the same value obtained directly.

Sample Mean Vectors and What They Estimate

For each group, the sample mean vector is $\bar Y_k=n_k^{-1}\sum_{i=1}^{n_k}Y_{ki}$. Under the independence and finite second-moment assumptions stated above, it estimates $\mu_k$ without bias, since linearity of expectation gives $E[\bar Y_k]=n_k^{-1}\sum_i E[Y_{ki}]=\mu_k$. Independence within group gives

\operatorname{Cov}(\bar Y_k) = \frac{1}{n_k^2}\sum_{i=1}^{n_k}\operatorname{Cov}(Y_{ki}) = \frac{\Sigma_k}{n_k}.

The covariance terms between distinct observations vanish because those observations are independent. Independence across groups likewise means that the stacked sample mean has covariance with diagonal blocks $\Sigma_1/n_1,\ldots,\Sigma_g/n_g$ and zero off-diagonal blocks. In particular, specifying the mean hypothesis does not require replacing these group-specific covariance matrices by a common one.

Common Mistakes and Why the Parameterization Matters

Confusing a sample difference with a population hypothesis. A nonzero difference between observed sample mean vectors is a calculated quantity. The hypothesis is an equation about the unknown population vectors.
Using weights that do not sum to zero but calling the result a contrast. Such a weighted sum changes if a common vector is added to every group mean. It may still define a linear function of means, but it does not isolate differences among groups.
Forgetting the outcome dimension. With $p$ outcomes, one group comparison is a $p$-vector equation, not a single scalar equation. A matrix with $q$ rows of group weights produces $q$ such vectors.
Assuming equal covariance matrices are part of the mean parameterization. The vector $\theta$ records only the group means. The covariance matrices $\Sigma_k$ can be distinct without changing how equality or planned comparisons are written.

Linear parameterization makes the scientific statement visible: the rows of a coefficient matrix specify which groups are combined, while each resulting vector equation specifies what is asserted across the outcomes. It also separates the question being asked about means from the covariance structure needed for later inference.

Check Your Understanding

Use the group-ordering convention and distinguish a population equation from a sample calculation.

For $g=4$ groups and $p=3$ outcomes, what is the length of $\theta$, and what is the dimension of $(C\otimes I_p)\theta$ if $C$ has two rows?
Write the coefficient rows that compare groups 1, 2, and 3 separately with group 4. How many scalar equations does the resulting equality hypothesis contain when $p=2$?
For four groups, determine whether $c=(1,-1,1,-1)'$ defines a contrast. Explain what happens to its value if the same vector is added to all four group means.
Express the hypothesis that the difference $\mu_1-\mu_2$ equals a specified vector $r$ using a coefficient row and the stacked parameter.
Why does calculating $\bar Y_1-\bar Y_2$ not, by itself, determine whether $\mu_1=\mu_2$?