Testing Estimability With the Row Space

Why Test a Contrast Directly?

When columns of a design matrix are linearly dependent, distinct coefficient vectors can produce exactly the same linear predictor. The coefficients may therefore fail to be individually determined, even though some combinations of them are determined. The practical question is not only whether the matrix is rank deficient, but whether a particular scientific contrast is still recoverable.

The row space gives a direct test. A contrast is estimable exactly when its coefficient vector can be written as a linear combination of the rows of the design matrix. This tutorial develops that criterion, proves its equivalence to the definition of estimability, and applies it to small rank-deficient designs.

Definition: Let \(X\) be an \(n\times p\) design matrix, \(\beta\in\mathbb{R}^p\) the coefficient vector, and \(\eta=X\beta\) the linear predictor. A coefficient contrast \(c^T\beta\), for a specified \(c\in\mathbb{R}^p\), is estimable if it has the same value for every coefficient vector that gives the same linear predictor. Equivalently, whenever \(X\beta=X\beta'\), estimability requires \(c^T\beta=c^T\beta'\). The row space of \(X\) is the subspace of \(\mathbb{R}^p\) spanned by its rows.

This is a structural property of the design: it says whether the contrast is determined by \(X\beta\). In a GLM whose response distribution depends on \(\beta\) through the linear predictor, coefficient vectors giving the same \(\eta\) also give the same model distribution. Estimability does not, by itself, guarantee a precise estimate or resolve other issues such as separation.

The Row-Space Test

Write the rows of \(X\) as \(x_1^T,\ldots,x_n^T\), where each \(x_i\in\mathbb{R}^p\). The row space consists of all vectors \(\sum_{i=1}^n a_i x_i\), with \(a\in\mathbb{R}^n\). Consequently, a simple way to check whether \(c\) is in the row space is to ask whether there is an \(a\) satisfying \(X^Ta=c\).

Theorem: For any \(X\in\mathbb{R}^{n\times p}\) and \(c\in\mathbb{R}^p\), the contrast \(c^T\beta\) is estimable if and only if \(c\) belongs to the row space of \(X\). Equivalently, it is estimable if and only if the linear system \(X^Ta=c\) has a solution \(a\in\mathbb{R}^n\).

Proof. First suppose that \(c\) is in the row space of \(X\). By the definition of row space, there is an \(a\in\mathbb{R}^n\) such that \(c=X^Ta\). If \(X\beta=X\beta'\), then

c^T\beta=(X^Ta)^T\beta=a^TX\beta =a^TX\beta'=(X^Ta)^T\beta'=c^T\beta'.

Thus the contrast has the same value for any two coefficient vectors producing the same linear predictor, so it is estimable.

Conversely, suppose \(c^T\beta\) is estimable. Every vector \(h\) in the null space of \(X\), defined as \(\operatorname{Null}(X)=\{h\in\mathbb{R}^p:Xh=0\}\), can be added to any coefficient vector without changing its linear predictor: \(X(\beta+h)=X\beta\). Estimability therefore implies \(c^T(\beta+h)=c^T\beta\), so \(c^Th=0\) for every \(h\in\operatorname{Null}(X)\). The row space of \(X\) is the orthogonal complement of \(\operatorname{Null}(X)\), so \(c\) belongs to the row space. Hence \(X^Ta=c\) has a solution. \(\square\)

The proof uses the standard finite-dimensional identity that the row space of \(X\) equals the orthogonal complement of its null space. The next result verifies this identity by a dimension argument; it also gives an equivalent test that can be useful for understanding why a contrast fails.

Result: A vector \(c\in\mathbb{R}^p\) is in the row space of \(X\) if and only if \(c^Th=0\) for every \(h\in\operatorname{Null}(X)\).

Proof. Every row of \(X\) is orthogonal to every \(h\in\operatorname{Null}(X)\), since the \(i\)th coordinate of \(Xh\) is the inner product of the \(i\)th row with \(h\), and \(Xh=0\). Thus the row space is contained in \(\operatorname{Null}(X)^\perp\). If \(r=\operatorname{rank}(X)\), the row space has dimension \(r\). By rank-nullity, \(\operatorname{Null}(X)\) has dimension \(p-r\), so its orthogonal complement in \(\mathbb{R}^p\) has dimension \(p-(p-r)=r\). A subspace contained in another subspace of the same finite dimension must equal it. Therefore the row space is \(\operatorname{Null}(X)^\perp\), proving the claim. \(\square\)

The two tests express the same geometry in different ways. The row-space test asks whether \(c\) is a row combination. The null-space test asks whether every coefficient change invisible to \(X\) also leaves the contrast unchanged.

Worked Applications

Worked Example: A Contrast From Two Identical Columns

Let

X=\begin{pmatrix}1&1\\2&2\end{pmatrix}, \qquad \beta=\begin{pmatrix}\beta_0\\\beta_1\end{pmatrix}.

The rows are \((1,1)\) and \((2,2)\), so the row space is \(\{(t,t):t\in\mathbb{R}\}\). Consider \(c=(1,1)^T\), which targets \(\beta_0+\beta_1\). It is in the row space: choosing \(a=(1,0)^T\) gives

X^Ta= \begin{pmatrix}1&2\\1&2\end{pmatrix} \begin{pmatrix}1\\0\end{pmatrix} =\begin{pmatrix}1\\1\end{pmatrix}=c.

Indeed, the first component of \(\eta=X\beta\) is \(\eta_1=\beta_0+\beta_1\). The sum is estimable even though the two coefficients are not separately determined.

Now consider \(d=(1,-1)^T\), which targets \(\beta_0-\beta_1\). A vector in the row space must have equal coordinates, whereas \(d\) does not. Thus this contrast is not estimable. The null-space check agrees: \(h=(1,-1)^T\) satisfies \(Xh=0\), but \(d^Th=2\ne0\). Moving from \(\beta\) to \(\beta+h\) leaves \(\eta\) unchanged while changing the contrast.

Worked Example: Testing Individual Coefficients in a Rank-Deficient Design

Consider

X=\begin{pmatrix} 1&0&1\\ 1&1&2\\ 1&2&3 \end{pmatrix}.

The third column is the sum of the first and second columns. A null-space vector is \(h=(-1,-1,1)^T\), since

Xh= \begin{pmatrix} -1+1\\ -1-1+2\\ -1-2+3 \end{pmatrix} =\begin{pmatrix}0\\0\\0\end{pmatrix}.

The null space is one-dimensional: the first two columns are linearly independent, so \(X\) has rank \(2\), and rank-nullity gives nullity \(3-2=1\). Therefore a contrast \(c=(c_0,c_1,c_2)^T\) is estimable precisely when it is orthogonal to \(h\):

c^Th=-c_0-c_1+c_2=0, \qquad\text{or equivalently}\qquad c_2=c_0+c_1.

For \(c=(1,0,1)^T\), which targets \(\beta_0+\beta_2\), the condition holds. It is also explicitly a row of \(X\): taking \(a=(1,0,0)^T\) gives \(X^Ta=(1,0,1)^T\). Hence \(\beta_0+\beta_2=\eta_1\) is estimable.

For \(\beta_0\) alone, use \(c=(1,0,0)^T\). The condition would require \(0=1+0\), which is false, so \(\beta_0\) alone is not estimable. For \(\beta_1\) alone, \(c=(0,1,0)^T\) also fails: it would require \(0=0+1\). Thus \(\beta_1\) alone is not estimable either. In each case, it is the specified contrast that has been tested.

Worked Example: A Row Combination Gives the Contrast

Let

X=\begin{pmatrix}1&0&1\\0&1&1\end{pmatrix}, \qquad \eta=\begin{pmatrix}\eta_1\\\eta_2\end{pmatrix}.

The row space consists of vectors of the form \((a,b,a+b)^T\). Take \(c=(2,-1,1)^T\). Its third coordinate equals the sum of its first two, so it lies in the row space. More explicitly, \(a=(2,-1)^T\) gives

X^Ta= \begin{pmatrix}1&0\\0&1\\1&1\end{pmatrix} \begin{pmatrix}2\\-1\end{pmatrix} =\begin{pmatrix}2\\-1\\1\end{pmatrix}=c.

Consequently, the target contrast can be written directly in terms of the linear predictor:

2\beta_0-\beta_1+\beta_2 =a^TX\beta=2\eta_1-\eta_2.

As a numerical check, for \(\beta=(2,3,5)^T\), the predictor is \(\eta=(7,8)^T\), and both expressions equal \(6\): \(2(2)-3+5=6\) and \(2(7)-8=6\). In contrast, \(c=(1,1,1)^T\) is not in the row space, since its third coordinate \(1\) is not the sum \(1+1=2\). Its target changes along the null direction \((-1,-1,1)^T\), so it is not estimable.

How to Carry Out the Test

For an exact, hand-worked calculation, solve \(X^Ta=c\). If a solution exists, \(c^T\beta=a^T\eta\), which exhibits the estimable contrast as a combination of components of the linear predictor. If the system is inconsistent, the contrast is not estimable. Equivalently, find a basis for \(\operatorname{Null}(X)\) and check that \(c\) is orthogonal to every basis vector.

Specify the target.
Write the contrast as \(c^T\beta\) and list the coordinates of \(c\) in the same order as the columns of \(X\).

Test row-space membership.
Form \(X^Ta=c\) and solve for \(a\). A solution means the contrast is estimable; an inconsistent system means it is not.

Interpret the result.
If a solution is found, use \(c^T\beta=a^T\eta\) to state what combination of the linear predictor equals the target.

With rounded or numerically computed matrices, a numerical rank tolerance can affect whether a system is treated as consistent. For a stated exact design, however, the criterion is exact row-space membership, not whether a software package happens to return a coefficient vector under a particular generalized inverse.

Common Mistake: Confusing a Fitted Predictor With Every Coefficient

A common error is to see that \(X\beta\) is determined and conclude that every coefficient, or every contrast, must therefore be determined. That does not follow. In the second example, the linear predictor is unchanged when a multiple of \((-1,-1,1)^T\) is added to \(\beta\). The contrast \(\beta_0+\beta_2\) remains unchanged under that move, but \(\beta_0\) alone does not.

The correction is to test the actual contrast vector. For each proposed \(c\), either solve \(X^Ta=c\) or check \(c^Th=0\) for a basis of the null space. Do not infer that a named coefficient is estimable merely because another coefficient, or a fitted linear predictor, is.

Key takeaway: A contrast \(c^T\beta\) is estimable exactly when \(c\) is a row combination of \(X\), equivalently when it is orthogonal to every null-space direction. Rank deficiency rules out some coefficient distinctions, but it does not rule out all scientifically meaningful contrasts.

Check Your Understanding

For each question, identify the target contrast before applying the test.

If \(X\) has rows \((1,1)\) and \((3,3)\), is \(\beta_0+\beta_1\) estimable? Give a row combination that verifies your answer.
For the same matrix, test whether \(\beta_0\) alone is estimable by checking a null-space vector.
Explain why a solution to \(X^Ta=c\) proves that \(c^T\beta\) is unchanged whenever \(X\beta\) is unchanged.
For the matrix in the second worked example, is \(2\beta_0+\beta_1+3\beta_2\) estimable? Apply the condition \(c_2=c_0+c_1\).
In one sentence, distinguish the structural meaning of estimability from the precision of an estimate.