Estimable Linear Functions Under Rank Deficiency

Why Some Coefficient Functions Remain Determined

When a design matrix is rank deficient, different coefficient vectors can produce exactly the same linear predictor. This means the individual coefficients are not uniquely determined by the predictor. It does not follow that every function of those coefficients is ambiguous: some combinations have the same value for every compatible coefficient vector. Estimability names this invariance.

Let \(X\) be a fixed \(n \times p\) design matrix, let \(\beta \in \mathbb{R}^p\) be a coefficient vector, and write the linear predictor as \(\eta=X\beta\). For a particular \(\eta\) that can be written in this form, consider all coefficient vectors that produce it. The question here is whether a specified linear function has one value across that entire set.

Definition: For a fixed design matrix \(X\), a linear function \(c^\mathsf{T}\beta\), where \(c \in \mathbb{R}^p\) is specified, is estimable if it has the same value for every coefficient vector \(\beta\) satisfying \(X\beta=\eta\), for any attainable \(\eta\). Equivalently, for each attainable \(\eta\), all coefficient vectors that produce that \(\eta\) give the same value of \(c^\mathsf{T}\beta\). A specified linear combination is sometimes called a contrast; the key issue here is whether that combination is invariant, not whether its coefficients meet a separate sum-to-zero convention.

The null space of \(X\) is the set of vectors \(v\) satisfying \(Xv=0\). If \(\beta_0\) produces \(\eta\), then so does \(\beta_0+v\) for every \(v\) in the null space, because \(X(\beta_0+v)=X\beta_0+Xv=\eta\). Conversely, any other coefficient vector producing \(\eta\) differs from \(\beta_0\) by a vector in the null space. Thus the full set of compatible coefficients is \(\beta_0+\operatorname{Null}(X)\). This gives a direct way to check whether a linear function is invariant over that set.

The Invariance Criterion

Theorem: A linear function \(c^\mathsf{T}\beta\) is estimable for \(X\) if and only if \(c^\mathsf{T}v=0\) for every \(v\) satisfying \(Xv=0\).

Proof. First suppose that \(c^\mathsf{T}\beta\) is estimable. Take any \(v\) satisfying \(Xv=0\), and choose any coefficient vector \(\beta_0\). The vectors \(\beta_0\) and \(\beta_0+v\) produce the same predictor. Estimability therefore gives

c^\mathsf{T}(\beta_0+v)=c^\mathsf{T}\beta_0.

Subtracting \(c^\mathsf{T}\beta_0\) from both sides yields \(c^\mathsf{T}v=0\).

Now suppose \(c^\mathsf{T}v=0\) for every \(v\) satisfying \(Xv=0\). Let \(\beta_1\) and \(\beta_2\) be any two coefficient vectors with the same predictor: \(X\beta_1=X\beta_2\). Then \(X(\beta_1-\beta_2)=0\), so \(\beta_1-\beta_2\) is in the null space. By the supposition,

c^\mathsf{T}\beta_1-c^\mathsf{T}\beta_2 =c^\mathsf{T}(\beta_1-\beta_2)=0.

Hence the two coefficient vectors give the same value of the linear function. Since the pair was arbitrary, the function is estimable. \(\square\)

This criterion is about invariance, not about finding one preferred coefficient vector. If a function is estimable, its value is fixed across the compatible coefficients. If it is not estimable, at least one null-space direction changes its value while leaving the predictor unchanged. A fitted coefficient vector may still be reported under a chosen convention, but that convention does not make a non-invariant function estimable.

Worked Example: A Sum Is Determined but an Individual Coefficient Is Not

Let

X= \begin{pmatrix} 1&1\\ 1&1 \end{pmatrix}, \qquad \eta= \begin{pmatrix} 3\\ 3 \end{pmatrix}.

The equation \(X\beta=\eta\) requires \(\beta_1+\beta_2=3\). For example, both \(\beta=(3,0)^\mathsf{T}\) and \(\beta=(0,3)^\mathsf{T}\) produce this \(\eta\).

For the sum, use \(c=(1,1)^\mathsf{T}\). Both solutions give \(c^\mathsf{T}\beta=3\). More generally, every solution has \(\beta_1+\beta_2=3\), so this function is estimable.

For the first coefficient, use \(c=(1,0)^\mathsf{T}\). The two displayed solutions give \(c^\mathsf{T}\beta=3\) and \(c^\mathsf{T}\beta=0\). Therefore \(\beta_1\) is not estimable. The same reasoning shows that \(\beta_2\) is not estimable, even though their sum is.

Checking a Contrast Across All Compatible Coefficients

For a concrete question, begin with one solution \(\beta_0\) of \(X\beta=\eta\). Every compatible vector is \(\beta_0+v\), where \(Xv=0\). The specified function takes the value

c^\mathsf{T}(\beta_0+v)=c^\mathsf{T}\beta_0+c^\mathsf{T}v.

Thus it is enough to ask whether the change \(c^\mathsf{T}v\) is zero for every null-space direction. If some such direction changes the function, that direction supplies two coefficient vectors with the same predictor but different function values.

Worked Example: An Estimable Difference and a Non-Estimable Difference

Consider

X= \begin{pmatrix} 1&0&1\\ 0&1&1 \end{pmatrix}, \qquad \eta= \begin{pmatrix} 4\\ 5 \end{pmatrix}.

The equations for compatible coefficients are \(\beta_1+\beta_3=4\) and \(\beta_2+\beta_3=5\). Setting \(\beta_3=t\) gives every solution:

\beta= \begin{pmatrix} 4-t\\ 5-t\\ t \end{pmatrix}.

The function \(\beta_1-\beta_2\) is \((4-t)-(5-t)=-1\) for every \(t\), so it is estimable for this design. In particular, the compatible vectors at \(t=0\) and \(t=1\) are \((4,5,0)^\mathsf{T}\) and \((3,4,1)^\mathsf{T}\); both give the difference \(-1\).

In contrast, \(\beta_1-\beta_3=(4-t)-t=4-2t\) changes with \(t\). At \(t=0\) it equals \(4\), and at \(t=1\) it equals \(2\), although both coefficient vectors produce \(\eta=(4,5)^\mathsf{T}\). This function is not estimable. Its coefficients sum to zero, but that fact alone does not ensure estimability under rank deficiency.

The definition can also be expressed without choosing a particular solution: for each attainable \(\eta\), the function must assign one value to the whole set of coefficients producing \(\eta\). When this holds, that value is determined by \(\eta\) alone. This is a precise statement about the linear predictor; by itself, it is not a claim that the value has been calculated from a response sample or that it has any particular sampling uncertainty.

Consequence: If \(c^\mathsf{T}\beta\) is estimable, then for every attainable \(\eta\) there is a well-defined value \(g(\eta)\): choose any \(\beta\) satisfying \(X\beta=\eta\) and set \(g(\eta)=c^\mathsf{T}\beta\). The choice of \(\beta\) does not affect \(g(\eta)\).

To verify this consequence, suppose \(\beta_1\) and \(\beta_2\) both satisfy \(X\beta=\eta\). By estimability, \(c^\mathsf{T}\beta_1=c^\mathsf{T}\beta_2\). Therefore the proposed value \(g(\eta)\) is independent of which solution is chosen. No uniqueness of the full coefficient vector is required.

Worked Example: A Proportional-Column Design

Let the design have three rows and two columns:

X= \begin{pmatrix} 1&2\\ 2&4\\ 0&0 \end{pmatrix}, \qquad \eta= \begin{pmatrix} 5\\ 10\\ 0 \end{pmatrix}.

The predictor equations reduce to \(\beta_1+2\beta_2=5\); the second row gives the same equation multiplied by \(2\), and the third gives \(0=0\). Write \(\beta_2=t\). All compatible coefficients are

\beta= \begin{pmatrix} 5-2t\\ t \end{pmatrix}.

For \(c=(2,4)^\mathsf{T}\), the function is

c^\mathsf{T}\beta=2(5-2t)+4t=10.

It is constant for every \(t\), so this function is estimable. For \(c=(1,0)^\mathsf{T}\), the function is \(5-2t\). It equals \(5\) at \(t=0\) and \(1\) at \(t=2\); both vectors, \((5,0)^\mathsf{T}\) and \((1,2)^\mathsf{T}\), produce the stated \(\eta\). Thus \(\beta_1\) is not estimable. The example shows why the question is not whether a particular computed coefficient vector gives a number, but whether all compatible vectors give the same number.

Common Pitfall and Practical Importance

Wrong reasoning: “The coefficient vector is not unique, so no linear function of its coefficients can be determined.” This treats uncertainty about the whole vector as if it necessarily affected every combination. The correction is to check how the particular function changes along null-space directions. In the first worked example, the vector is not unique, but \(\beta_1+\beta_2\) is fixed at \(3\).

The reverse mistake is also common: calculate a function at one convenient solution and call it estimable because it has a numerical value. A number from one selected solution does not establish invariance. For example, \(\beta_1\) equals \(3\) at \((3,0)^\mathsf{T}\) in the first example, but equals \(0\) at another coefficient vector producing exactly the same predictor. Always compare across all compatible vectors, or use the null-space criterion.

Estimability matters because interpretation often concerns combinations rather than isolated coefficients. Under rank deficiency, a model may determine a sum, difference, or other linear combination even when it cannot uniquely distinguish the coefficients that make up that combination. The design, not the label “contrast” or the choice of a coefficient solution, decides which linear functions are invariant.

Check Your Understanding

For each question, focus on whether the specified function stays fixed across all coefficient vectors producing the same linear predictor.

For the matrix in the first worked example, is \(2\beta_1+2\beta_2\) estimable? Give its value when \(\eta=(3,3)^\mathsf{T}\).
For the matrix in the second worked example, write all compatible coefficient vectors using a free parameter and determine whether \(\beta_1+\beta_2\) is estimable.
Explain why showing two compatible coefficient vectors with different values of \(c^\mathsf{T}\beta\) is enough to establish non-estimability.
For \(X\) with a nonzero null-space vector \(v\), what condition must \(c\) satisfy along that direction for \(c^\mathsf{T}\beta\) to be estimable?
Why does calculating \(c^\mathsf{T}\beta\) at one solution of \(X\beta=\eta\) not, by itself, show that the function is estimable?

Estimable Linear Functions Under Rank Deficiency

What You'll Learn