Matrix Form of a General Linear Hypothesis

Why Write Joint Restrictions in Matrix Form?

A question about several regression coefficients is often a single joint question. For example, a researcher may ask whether two slopes are both zero, whether two slopes are equal, or whether a combination of slopes equals a specified value. Writing each statement separately is possible, but a matrix equation makes the collection of restrictions explicit and provides a direct way to identify how many are genuinely independent.

Consider the linear model with coefficient vector \(\beta\in\mathbb{R}^p\). A linear restriction is a statement about a weighted sum of these coefficients. For instance, with \(\beta=(\beta_0,\beta_1,\beta_2)^T\), the statement \(\beta_1-\beta_2=0\) is a linear restriction. Several such statements can be collected into a matrix equation.

Definition: A general linear hypothesis is a null hypothesis of the form \(H_0:L\beta=r\), where \(L\) is a fixed \(q\times p\) matrix, \(\beta\in\mathbb{R}^p\) is the coefficient vector, and \(r\in\mathbb{R}^q\) is a fixed vector. The \(i\)th row of \(L\) specifies the coefficients in the \(i\)th restriction; the corresponding entry of \(r\) is its target value. The restrictions are independent when the rows of \(L\) are linearly independent.

The displayed equation represents \(q\) scalar equations, one for each row of \(L\). The number \(q\) is the number of rows written down, but it need not be the number of distinct constraints: some rows may be linear combinations of others. The rank of \(L\), denoted \(\operatorname{rank}(L)\), counts the linearly independent rows.

Building the Restriction Matrix

To form \(L\), first fix the order of the coefficients in \(\beta\). Then write each restriction in that order, using zero for coefficients that do not appear. For example, if \(\beta=(\beta_0,\beta_1,\beta_2,\beta_3)^T\), the restriction \(\beta_1-\beta_3=2\) has row \((0,1,0,-1)\) and target 2. The restriction \(\beta_0+\beta_2=0\) has row \((1,0,1,0)\) and target 0.

The right-hand side matters as much as the rows of \(L\). A row dependence means that one left-hand side is determined by others; for it to be a redundant equation, its target must be determined by the corresponding targets in exactly the same way. If the left-hand sides are dependent but the targets disagree, the equations cannot all hold.

Worked Example: Testing Three Restrictions on Four Coefficients

Let \(\beta=(\beta_0,\beta_1,\beta_2,\beta_3)^T\). Suppose the joint null hypothesis is \(\beta_1-\beta_2=0\), \(\beta_1+\beta_2=0\), and \(\beta_3=0\). Write the hypothesis in matrix form and determine its rank.

First restriction. The coefficients of \((\beta_0,\beta_1,\beta_2,\beta_3)\) in \(\beta_1-\beta_2\) are \((0,1,-1,0)\).

Second restriction. The coefficients in \(\beta_1+\beta_2\) are \((0,1,1,0)\).

Third restriction. The coefficients in \(\beta_3\) are \((0,0,0,1)\). Therefore,

L= \begin{bmatrix} 0&1&-1&0\\ 0&1&1&0\\ 0&0&0&1 \end{bmatrix}, \qquad r= \begin{bmatrix} 0\\0\\0 \end{bmatrix}, \qquad H_0:L\beta=r.

The first two rows are independent: if \(a(0,1,-1,0)+b(0,1,1,0)=0\), then \(a+b=0\) and \(-a+b=0\), which imply \(a=b=0\). The third row is also independent of the first two because it is the only row with a nonzero fourth entry. Thus \(\operatorname{rank}(L)=3\), so these are three independent restrictions. In this particular case, adding and subtracting the first two equations shows that the null also implies \(\beta_1=0\) and \(\beta_2=0\); the matrix form retains the original joint statements without changing their meaning.

Redundant Rows and Compatible Targets

A restriction is redundant if it adds no new condition to the equations already present. Algebraically, suppose a row of \(L\) is a linear combination of other rows. The same linear combination must hold among the corresponding entries of \(r\) for that equation to be redundant. Row rank alone identifies dependence among left-hand sides; it does not establish that the full system \(L\beta=r\) is consistent.

Theorem: Let the rows of \(L\) have rank \(k\), and choose \(k\) independent rows as a basis. The system \(L\beta=r\) is consistent exactly when every row dependence among \(L\)'s rows also holds among the corresponding entries of \(r\). When it is consistent, the \(k\) basis equations imply all the remaining equations, so the hypothesis has \(k\) independent restrictions.

Proof. Denote the \(k\) chosen rows by the rows of \(B\), and their target entries by \(s\). For any remaining row \(L_j\), there are coefficients \(a_j\) such that \(L_j=a_j^TB\), because the chosen rows span the row space of \(L\). If the targets satisfy \(r_j=a_j^Ts\), then every solution of \(B\beta=s\) also satisfies

L_j\beta=a_j^TB\beta=a_j^Ts=r_j.

Thus each remaining equation follows from the basis equations. Conversely, if \(L_j=a_j^TB\) but \(r_j\ne a_j^Ts\), any \(\beta\) satisfying the basis equations would have to satisfy both \(L_j\beta=a_j^Ts\) and \(L_j\beta=r_j\), which is impossible. This proves both the consistency criterion and the count of independent restrictions. \(\square\)

Worked Example: Finding Two Redundant Restrictions

Let \(\beta=(\beta_0,\beta_1,\beta_2)^T\), and consider

L= \begin{bmatrix} 1&1&0\\ 0&1&1\\ 1&2&1 \end{bmatrix}, \qquad r= \begin{bmatrix} 2\\-1\\1 \end{bmatrix}.

The third row of \(L\) is the sum of the first and second rows: \((1,2,1)=(1,1,0)+(0,1,1)\). The target entries obey the same relation: \(1=2+(-1)\). Thus the third equation is implied by the first two.

Explicitly, the first two equations are \(\beta_0+\beta_1=2\) and \(\beta_1+\beta_2=-1\). Adding them gives \(\beta_0+2\beta_1+\beta_2=1\), which is exactly the third equation. The first two rows are independent, so \(\operatorname{rank}(L)=2\). There are three written equations but only two independent restrictions.

For contrast, change the last entry of \(r\) from 1 to 0. The left-hand sides still satisfy “third equals first plus second,” but the targets would require \(0=2+(-1)=1\). No coefficient vector can satisfy all three equations. The system is inconsistent; it does not describe a nonempty null hypothesis.

Rank and Equivalent Ways to State a Hypothesis

Different matrices can encode the same set of restrictions. For example, one may multiply a restriction by a nonzero constant, or replace one equation by its sum with another equation, provided the target is transformed in the same way. More generally, an invertible \(q\times q\) matrix \(A\) gives an equivalent system:

L\beta=r \quad\Longleftrightarrow\quad AL\beta=Ar.

The equivalence follows because \(A\) has an inverse: multiplying the second equation by \(A^{-1}\) recovers the first. Such row operations preserve the solution set and preserve the rank. They are useful for exposing dependencies, but changing \(L\) without making the corresponding change to \(r\) can change the hypothesis.

When the system is consistent, \(\operatorname{rank}(L)\) is the number of independent linear restrictions, not necessarily the number of rows. This is the relevant count when describing the hypothesis. The matrix form alone does not specify a test statistic or its calibration; it identifies the null restrictions and their independent dimension.

Worked Example: Comparing Two Equivalent Matrix Forms

Suppose \(\beta=(\beta_0,\beta_1,\beta_2)^T\) and the restrictions are \(\beta_1=1\) and \(\beta_2=-2\). One representation is

L= \begin{bmatrix} 0&1&0\\ 0&0&1 \end{bmatrix}, \qquad r= \begin{bmatrix} 1\\-2 \end{bmatrix}.

Add the first equation to the second. This is the row operation represented by \(A=\begin{bmatrix}1&0\\1&1\end{bmatrix}\), which is invertible because its determinant is 1. The transformed system is

AL= \begin{bmatrix} 0&1&0\\ 0&1&1 \end{bmatrix}, \qquad Ar= \begin{bmatrix} 1\\-1 \end{bmatrix}.

The second transformed equation says \(\beta_1+\beta_2=-1\); together with \(\beta_1=1\), it gives \(\beta_2=-2\). Thus both matrix forms specify exactly the same set of coefficient vectors. The two rows remain independent, so both matrices have rank 2.

Estimability in a Rank-Deficient Model

In a model with a rank-deficient design matrix \(X\), the coefficient vector may not be uniquely determined by the mean vector \(X\beta\). A restriction on coefficients is interpretable as a property of the model's mean structure only when the relevant linear combination is estimable. For a row vector \(l^T\), the linear function \(l^T\beta\) is estimable exactly when \(l^T\) lies in the row space of \(X\), equivalently when \(l^Tv=0\) for every \(v\) in the null space of \(X\). For the whole hypothesis, each row of \(L\) must meet this condition if the restrictions are to be statements about identifiable linear functions of the mean.

This distinction matters because \(\beta\) and \(\beta+v\) produce the same mean whenever \(Xv=0\). If \(l^Tv\ne0\), the value of \(l^T\beta\) changes between those coefficient vectors even though the model's mean is unchanged. A coefficient restriction that depends on such a non-identifiable direction cannot be interpreted as a unique restriction on the fitted mean structure. In a full-column-rank design, every linear function of \(\beta\) is estimable; in a rank-deficient design, check estimability before interpreting the matrix hypothesis.

Common Mistakes

Counting rows instead of independent restrictions. A matrix with four rows can have rank two. Use \(\operatorname{rank}(L)\), not just the row count, to determine the number of independent left-hand sides.
Checking dependencies in \(L\) but ignoring \(r\). A dependent row makes a redundant equation only if its target obeys the same dependence. Otherwise the system is inconsistent.
Changing the left-hand sides without changing the targets. Row operations preserve the hypothesis only when applied to the entire equation, transforming both \(L\) and \(r\).
Assuming every coefficient statement is estimable. With a rank-deficient design, verify that each row of \(L\) lies in the row space of \(X\) before treating the restrictions as identifiable properties of the mean.
Confusing a joint null with separate decisions. \(L\beta=r\) states that all listed restrictions hold together. It is one joint hypothesis, even though it contains multiple scalar equations.

Key takeaway: Write each restriction in the coefficient order of \(\beta\), collect the rows into \(L\), and place their target values in \(r\). Then check the rank, verify that any row dependencies are matched by the targets, and—when the design is rank deficient—confirm that the restrictions are estimable.

Check Your Understanding

For each question, identify the matrix or rank reasoning needed before drawing a conclusion.

For \(\beta=(\beta_0,\beta_1,\beta_2)^T\), write \(H_0:\beta_1=0,\ \beta_1-\beta_2=3\) as \(L\beta=r\).
If the third row of \(L\) is the sum of its first two rows, what condition must the entries of \(r\) satisfy for the third equation to be redundant?
A \(5\times p\) restriction matrix has rank 3. If the system is consistent, how many independent restrictions does it express?
Why does multiplying \(L\) by an invertible matrix require multiplying \(r\) by that same matrix?
In a rank-deficient design, what condition on a row \(l^T\) makes \(l^T\beta\) estimable?