General Linear Hypotheses for Multivariate Responses

From a Scientific Question to a Coefficient Hypothesis

A multivariate regression can describe several response variables in one model. A scientific question may concern one coefficient for one response, or it may compare effects across responses or combine several coefficients. A general linear hypothesis gives these questions a common form: it specifies which linear combinations of the regression coefficients are required to take particular values.

Let $Y$ be an $n\times p$ response matrix, $X$ a fixed $n\times q$ design matrix, and $B$ a $q\times p$ coefficient matrix in the multivariate linear model $Y=XB+U$. The entry $b_{ij}$ is the coefficient of predictor $i$ for response $j$. The structure of a hypothesis concerns $B$; the covariance among response errors affects inference about that hypothesis, not its definition.

Definition: Write $\operatorname{vec}(B)$ for the $qp$-vector formed by stacking the columns of $B$, in order. A general linear hypothesis is a set of linear equality restrictions $$ H_0:\ R\operatorname{vec}(B)=r, $$ where $R$ is a specified $s\times qp$ matrix and $r$ is a specified $s$-vector. Its null parameter space is $\mathcal{B}_0=\{B:R\operatorname{vec}(B)=r\}$. The unrestricted parameter space is $\mathcal{B}=\mathbb{R}^{q\times p}$, and the standard two-sided alternative is $H_1:\ R\operatorname{vec}(B)\ne r$, or equivalently $B\in\mathcal{B}\setminus\mathcal{B}_0$.

The vectorization convention matters because it determines the locations of coefficients in $R\operatorname{vec}(B)$. A restriction is not a statement about the observed responses $Y$, nor is it generally a statement that every entry of $B$ is zero. It selects particular linear combinations of entries in the coefficient matrix.

For a nonempty null space, the vector $r$ must be compatible with $R$. If $R$ has $s$ independent rows, the restrictions are independent and impose $s$ constraints. If some rows are redundant, the number of effective constraints is the rank of $R$, not the number of written equations. The examples below use independent restrictions unless indicated otherwise.

Examples of Null and Alternative Spaces

The null hypothesis describes a subset of possible coefficient matrices. The alternative is the collection of all unrestricted coefficient matrices that fail at least one null restriction. In particular, for several restrictions, the standard alternative is not a collection of separate directional claims; it says that the restriction vector differs from its specified value.

Worked Example: Jointly Testing a Predictor Across Two Responses

Suppose a model has an intercept, a treatment indicator, and age as predictors, with two responses. Write the coefficient matrix as $$ B= \begin{pmatrix} b_{11}&b_{12}\\ b_{21}&b_{22}\\ b_{31}&b_{32} \end{pmatrix}. $$ The column-stacked vector is $$ \operatorname{vec}(B)=(b_{11},b_{21},b_{31},b_{12},b_{22},b_{32})'. $$

To ask whether treatment has no effect on either response, impose $b_{21}=0$ and $b_{22}=0$. Thus $$ R= \begin{pmatrix} 0&1&0&0&0&0\\ 0&0&0&0&1&0 \end{pmatrix}, \qquad r= \begin{pmatrix}0\\0\end{pmatrix}. $$ Multiplication gives $R\operatorname{vec}(B)=(b_{21},b_{22})'$, so the null space consists exactly of matrices with both treatment coefficients zero. The alternative consists of matrices for which at least one of those two coefficients is nonzero. It does not require both coefficients to be nonzero.

Worked Example: Equality of Slopes Across Responses

Let the predictors be an intercept and a treatment indicator, and let there be two responses: $$ B= \begin{pmatrix} b_{11}&b_{12}\\ b_{21}&b_{22} \end{pmatrix}, \qquad \operatorname{vec}(B)=(b_{11},b_{21},b_{12},b_{22})'. $$ Suppose the question is whether the treatment slopes are equal across the two responses. The restriction is $b_{21}-b_{22}=0$, represented by $$ R=\begin{pmatrix}0&1&0&-1\end{pmatrix}, \qquad r=0. $$

Indeed, $R\operatorname{vec}(B)=b_{21}-b_{22}$. Therefore, the null space is $\{B:b_{21}=b_{22}\}$, while the alternative is $\{B:b_{21}\ne b_{22}\}$. The intercepts $b_{11}$ and $b_{12}$ are unrestricted under both hypotheses. This is a comparison of effects, not a claim that either treatment slope is zero.

Worked Example: A Nonzero Linear Contrast

Use the same two-predictor, two-response arrangement. Consider the question whether twice the treatment slope for response 1 plus the treatment slope for response 2 equals 3. The coefficient restriction is $2b_{21}+b_{22}=3$, so $$ R=\begin{pmatrix}0&2&0&1\end{pmatrix}, \qquad r=3. $$

For $\operatorname{vec}(B)=(b_{11},b_{21},b_{12},b_{22})'$, multiplication yields $R\operatorname{vec}(B)=2b_{21}+b_{22}$, as required. Solving the null equation gives $b_{22}=3-2b_{21}$. The two intercepts and $b_{21}$ may vary freely, and $b_{22}$ is then determined. Under the alternative, $b_{22}\ne3-2b_{21}$. The constant $r$ need not be zero; a general linear hypothesis can specify any compatible target value.

Geometry and Number of Restrictions

The null space is affine: it is a translated linear subspace, rather than necessarily a subspace through the origin. This follows directly from solving the restriction equations.

Theorem: Suppose $R$ is an $s\times qp$ matrix and the equation $R\beta=r$ is consistent, where $\beta=\operatorname{vec}(B)$. If $k=\operatorname{rank}(R)$, then the null parameter space is an affine space of dimension $qp-k$. In particular, if $R$ has full row rank $s$, the restrictions reduce the dimension from $qp$ to $qp-s$.

Proof. Consistency means that there is at least one vector $\beta_0$ satisfying $R\beta_0=r$. For any vector $\beta$, $$ R\beta=r \quad\Longleftrightarrow\quad R(\beta-\beta_0)=0 \quad\Longleftrightarrow\quad \beta-\beta_0\in\ker(R). $$ Consequently, the full solution set is $\beta_0+\ker(R)$. Translation does not change dimension, and the rank-nullity theorem gives $\dim\ker(R)=qp-\operatorname{rank}(R)=qp-k$. This proves the stated dimension. $\square$

This result explains why redundant equations do not add restrictions. For example, writing both $b_{21}=0$ and $2b_{21}=0$ supplies two equations but only one independent condition. The rank of their coefficient matrix is one, so the null space loses one dimension, not two.

Coefficient Restrictions and the Mean Model

A coefficient hypothesis also determines a restricted set of possible mean matrices $XB$. To interpret it as a distinction between mean models, it matters whether different coefficient matrices can produce the same mean matrix. When $X$ has full column rank, they cannot.

Result: If $X$ has full column rank $q$, the mapping $B\mapsto XB$ is one-to-one. Therefore, a coefficient restriction defines a corresponding restricted set of mean matrices, and distinct coefficient matrices in the unrestricted model give distinct mean matrices.

Proof. Suppose $XB_1=XB_2$. Then $X(B_1-B_2)=0$. Full column rank implies that the null space of $X$ contains only the zero vector. Applying this fact to each column of $B_1-B_2$ gives $B_1-B_2=0$, hence $B_1=B_2$. Equivalently, $(X'X)^{-1}X'$ is a left inverse of $X$, since $(X'X)^{-1}X'X=I_q$. Thus $B$ can be recovered from the mean matrix $XB$ by multiplying on the left by this left inverse. $\square$

If $X$ is rank deficient, some coefficient directions are not identifiable from the mean matrix. In that case, a restriction on an arbitrary entry of $B$ need not describe an identifiable feature of the model. A hypothesis must concern estimable coefficient combinations, or the model must be parameterized to remove the nonidentifiability. This is why the algebraic form of a restriction and its interpretation under the design should be considered together.

Why the General Form Is Useful

The matrix $R$ is a bookkeeping device that turns a scientific question into explicit equations. It makes clear which coefficients are involved, what values are asserted under the null, and how many independent restrictions are present. It also allows a single hypothesis to cover several responses without pretending that they are unrelated. The covariance among responses does not change the restriction $R\operatorname{vec}(B)=r$, though it is important when calculating a test.

The next stage of analysis builds quantities for assessing such hypotheses from the fitted model. Here the essential task is to state the parameter space accurately; no test statistic or reference distribution is implied merely by writing down $R$ and $r$.

Common Mistakes

Confusing the null with one fitted matrix. The null does not generally claim that $B$ equals a particular estimated matrix. It specifies a set of coefficient matrices satisfying the stated equations.
Treating a joint alternative as “all effects are nonzero.” For a null setting several coefficients to zero, the standard alternative is that at least one is nonzero.
Ignoring the order in $\operatorname{vec}(B)$. If columns are stacked, coefficients for the first response come before those for the second. A row of $R$ must match that order.
Counting equations instead of independent restrictions. Redundant rows of $R$ do not add constraints; use $\operatorname{rank}(R)$.
Assuming a coefficient is identifiable when the design is rank deficient. The mean model may not distinguish coefficient matrices that differ in nonidentifiable directions.

Key takeaway: A general linear hypothesis specifies a null affine space through $R\operatorname{vec}(B)=r$, with the standard alternative consisting of all coefficient matrices outside that space. Careful vector ordering, rank, and identifiability keep the mathematical restriction aligned with the scientific question.

Check Your Understanding

For each question, identify the coefficient combination involved and describe the null and alternative spaces.

With two responses, how would you express the null that a predictor's coefficient is zero for both outcomes?
For two response-specific slopes, what is the difference between testing that they are equal and testing that both are zero?
If $R$ has four rows but rank three, how many independent restrictions does $R\operatorname{vec}(B)=r$ impose when it is consistent?
In a model with a rank-deficient design matrix, why might a restriction on an individual entry of $B$ fail to describe an identifiable feature?
Under the standard two-sided alternative, what does $R\operatorname{vec}(B)\ne r$ mean when the restriction vector has several components?