Estimability Conditions for Exact Contrast Tests

Why Estimability Comes Before Testing

In a full-rank linear model, each coefficient is identified by the design and the data. When the design matrix is rank deficient, different coefficient vectors can produce exactly the same mean response. A proposed contrast may still be identifiable—for example, a sum of coefficients may be identifiable even when its individual terms are not. The key question is therefore not whether the coefficients can each be estimated, but whether the particular linear combination being tested is determined by the model’s mean vector.

This distinction matters for exact \(t\) inference. The usual statistic requires a contrast estimate whose expectation is the contrast parameter under every parameter vector in the model. If the contrast is not determined by the mean vector, no such unbiased estimator exists, and different values of the proposed contrast can correspond to the same data distribution.

Definition: Consider the fixed-design linear model \(Y=X\beta+\varepsilon\), where \(X\) is a known \(n\times p\) matrix and \(\beta\in\mathbb{R}^p\). A linear contrast \(c'\beta\), for a specified vector \(c\in\mathbb{R}^p\), is estimable if there is a vector \(a\in\mathbb{R}^n\) such that \(E[a'Y]=c'\beta\) for every \(\beta\). Equivalently, \(X'a=c\) for some \(a\).

The definition concerns the model’s possible mean vectors, not the observed response values. It also does not require \(X\) to have full column rank. The equivalent condition \(X'a=c\) says that an estimable contrast can be recovered as a linear combination of the expected observations.

Three Equivalent Ways to Check

Let \(\mathcal{N}(X)=\{v:Xv=0\}\) be the null space of \(X\), and let \(\mathcal{R}(X')\) be the column space of \(X'\), also called the row space of \(X\). The fundamental orthogonality relation is \(\mathcal{R}(X')=\mathcal{N}(X)^\perp\). Thus estimability has three equivalent tests.

Theorem: A contrast \(c'\beta\) is estimable if and only if any (and hence all) of the following conditions hold:

\(c\in\mathcal{R}(X')\), the row space of \(X\);
\(c'v=0\) for every \(v\in\mathcal{N}(X)\);
there exists \(a\in\mathbb{R}^n\) such that \(X'a=c\).

Proof. By the definition of estimability, a contrast is estimable exactly when there is an \(a\) such that \(E[a'Y]=c'\beta\) for every \(\beta\). Since \(E[Y]=X\beta\), this equality is \(a'X\beta=c'\beta\) for every \(\beta\), which holds exactly when \(X'a=c\). The set of vectors \(X'a\) is \(\mathcal{R}(X')\), so this is equivalent to \(c\in\mathcal{R}(X')\). Finally, the row space is the orthogonal complement of the null space, so \(c\in\mathcal{R}(X')\) exactly when \(c'v=0\) for every \(v\) with \(Xv=0\). \(\square\)

The null-space condition gives the most direct interpretation. If \(v\in\mathcal{N}(X)\), then \(\beta\) and \(\beta+v\) produce the same mean, because \(X(\beta+v)=X\beta\). A contrast is estimable precisely when it has the same value at \(\beta\) and \(\beta+v\) for every such \(v\).

Worked Example: Duplicate Predictors

Suppose the two columns of a design matrix are identical:

X= \begin{pmatrix} 1&1\\ 2&2\\ 3&3 \end{pmatrix}.

Here \(X\beta\) depends on \(\beta\) only through \(\beta_1+\beta_2\). The vector \(v=(1,-1)'\) is in the null space, since \(Xv=0\). For \(c=(1,1)'\), \(c'v=1-1=0\), so \(\beta_1+\beta_2\) is estimable. Indeed, the first observation’s mean is \(\beta_1+\beta_2\), and \(a=(1,0,0)'\) satisfies \(X'a=(1,1)'=c\).

For \(c=(1,0)'\), \(c'v=1\ne0\), so \(\beta_1\) alone is not estimable. For instance, \((\beta_1,\beta_2)=(2,3)\) and \((3,2)\) give the same mean vector but different values of \(\beta_1\). The data cannot distinguish those coefficient values through this design.

Estimable Contrasts and Least Squares

When \(X\) is rank deficient, a least-squares coefficient vector need not be unique: adding any \(v\in\mathcal{N}(X)\) leaves the fitted values unchanged. For an estimable contrast, however, its value is the same for every least-squares solution. This is why testing an estimable contrast does not require identifying each coefficient separately.

Theorem: If \(c'\beta\) is estimable, then \(c'\hat\beta\) has the same value for every least-squares solution \(\hat\beta\) minimizing \(\|Y-Xb\|^2\). If \(c'\beta\) is not estimable, no linear unbiased estimator of it exists.

Proof. Any two least-squares solutions have the same fitted values, and their difference \(v\) satisfies \(Xv=0\). If \(c'\beta\) is estimable, the null-space condition gives \(c'v=0\). Therefore the two solutions give the same contrast value. Conversely, if no \(a\) satisfies \(X'a=c\), then \(E[a'Y]=a'X\beta\) cannot equal \(c'\beta\) for every \(\beta\) for any linear statistic \(a'Y\). Thus no linear unbiased estimator exists. \(\square\)

Worked Example: A Rank-Deficient Design With Two Identifiable Combinations

Consider a four-observation design whose third column is the sum of its first two:

X= \begin{pmatrix} 1&0&1\\ 0&1&1\\ 1&1&2\\ 2&0&2 \end{pmatrix}.

The vector \(v=(-1,-1,1)'\) satisfies \(Xv=0\). Therefore a contrast \(c'\beta\) is estimable exactly when \(-c_1-c_2+c_3=0\). The contrast \(\beta_1-\beta_2\), with \(c=(1,-1,0)'\), passes this check: \(-1+1+0=0\). An explicit observation-weight vector is \(a=(1,-1,0,0)'\), since direct multiplication gives \(X'a=(1,-1,0)'\). Thus \(a'Y=Y_1-Y_2\) is unbiased for \(\beta_1-\beta_2\).

By contrast, \(\beta_3\), with \(c=(0,0,1)'\), fails because \(c'v=1\). The reason is visible from the mean: the third predictor column duplicates the sum of the first two. The model’s mean depends on \(\beta_1+\beta_3\) and \(\beta_2+\beta_3\), but not on \(\beta_3\) separately.

Why an Estimable Contrast Has an Exact \(t\) Test

For exact normal-theory inference, assume \(X\) is fixed, \(\varepsilon\sim N(0,\sigma^2 I_n)\) with \(\sigma^2>0\), and the rank of \(X\) is \(r<n\). The residual degrees of freedom are \(\nu=n-r\), so the residual mean square is \(MSE=SSE/\nu\). These conditions ensure that the residual variance estimate has a scaled chi-square distribution and that the residual component is independent of the fitted component.

For an estimable contrast, choose \(a_0\in\mathcal{R}(X)\) satisfying \(X'a_0=c\). Such a vector exists: an initial solution \(a\) exists by estimability, and its projection onto \(\mathcal{R}(X)\) is also a solution. It is unique within \(\mathcal{R}(X)\), since a vector in both \(\mathcal{R}(X)\) and the null space of \(X'\) must be zero. The least-squares estimate of the contrast is \(a_0'Y\), and its variance is \(\sigma^2 a_0'a_0\).

Theorem: Under the fixed-design Gaussian model above, let \(c\ne0\) be estimable. To test \(H_0:c'\beta=\psi\), the statistic \[ T=\frac{a_0'Y-\psi}{\sqrt{MSE\,a_0'a_0}}, \qquad X'a_0=c,\quad a_0\in\mathcal{R}(X), \] has a \(t_\nu\) distribution under \(H_0\). The vector \(a_0\) is the unique vector in \(\mathcal{R}(X)\) satisfying the displayed constraint.

Proof. Because \(X'a_0=c\), the numerator before studentization has expectation \(a_0'X\beta=c'\beta\). Under \(H_0\), \(a_0'Y-\psi=a_0'\varepsilon\), which is normal with mean zero and variance \(\sigma^2a_0'a_0\). Let \(P_X\) be the orthogonal projection onto \(\mathcal{R}(X)\), and let \(M_X=I-P_X\). Since \(a_0\in\mathcal{R}(X)\), \(a_0'M_X=0\). The residual sum of squares is \(SSE=Y'M_XY=\varepsilon'M_X\varepsilon\). Under Gaussian errors, the orthogonal components \(P_X\varepsilon\) and \(M_X\varepsilon\) are independent. Consequently, \(a_0'\varepsilon\) is independent of \(SSE\). The projection \(M_X\) has rank \(\nu\), so \(SSE/\sigma^2\sim\chi^2_\nu\). Thus \(MSE/\sigma^2\) is an independent \(\chi^2_\nu/\nu\) variable. The ratio of an independent standard normal variable to the square root of an independent \(\chi^2_\nu/\nu\) variable has a \(t_\nu\) distribution, giving the result. \(\square\)

Worked Example: An Exact Test for an Estimable Sum

Return to the duplicate-column design with four observations and \(x=(1,2,3,4)'\):

X= \begin{pmatrix} 1&1\\ 2&2\\ 3&3\\ 4&4 \end{pmatrix}, \qquad Y= \begin{pmatrix} 2\\3\\5\\6 \end{pmatrix}.

The estimable contrast is \(c'\beta=\beta_1+\beta_2\). Write \(\gamma=\beta_1+\beta_2\), so the model mean is \(x\gamma\). The least-squares estimate is

\hat\gamma=\frac{x'Y}{x'x} =\frac{1(2)+2(3)+3(5)+4(6)}{1^2+2^2+3^2+4^2} =\frac{47}{30}.

A vector in \(\mathcal{R}(X)\) satisfying \(X'a_0=(1,1)'\) is \(a_0=x/30\), because \(x'x=30\). Hence \(a_0'a_0=1/30\), and the contrast estimate is \(a_0'Y=47/30\). The residual sum of squares can be calculated by subtracting the fitted sum of squares from \(Y'Y\):

SSE=Y'Y-\frac{(x'Y)^2}{x'x} =74-\frac{47^2}{30} =\frac{11}{30}, \qquad MSE=\frac{SSE}{4-1}=\frac{11}{90}.

For \(H_0:\beta_1+\beta_2=0\), the exact statistic is therefore

T= \frac{47/30}{\sqrt{(11/90)(1/30)}} =\frac{47/30}{\sqrt{11/2700}} \approx 24.55.

The residual degrees of freedom are \(4-\operatorname{rank}(X)=4-1=3\), so under the null this statistic has a \(t_3\) distribution. The individual coefficients are not separately estimable, but their sum is, and its exact test is well-defined.

Why a Nonestimable Contrast Cannot Have This Test

A nonestimable contrast does not merely have a difficult standard error. Its value is not identified by the distribution of the data. If \(c'v\ne0\) for some \(v\in\mathcal{N}(X)\), moving from \(\beta\) to \(\beta+tv\) leaves the distribution of \(Y\) unchanged for every real \(t\), while changing \(c'\beta\) by \(t c'v\). By choosing \(t\), that contrast can take any real value without changing the data distribution.

In particular, for any parameter vector under an alternative value of \(c'\beta\), one can move along a null-space direction to obtain a parameter vector satisfying a specified null value, with exactly the same distribution of \(Y\). A test that controls its rejection probability at level \(\alpha\) for every parameter under the null must then have rejection probability at most \(\alpha\) at that indistinguishable alternative as well. There is no nontrivial test that separates those values uniformly over the model.

Worked Example: The Same Distribution Under Different Contrast Values

In the two-column duplicate design, suppose the target is \(H_0:\beta_1=0\). The parameter vectors \(\beta^{(0)}=(0,2)'\) and \(\beta^{(1)}=(1,1)'\) give the same sum of coefficients, namely \(2\), and therefore the same mean vector \(X\beta=(2,4,6)'\). They have different values of \(\beta_1\): zero and one.

With the same Gaussian error distribution, the full distributions of \(Y\) under these two parameter vectors are identical. Thus any test’s rejection probability is identical under the null parameter vector and this alternative parameter vector. A test that has level \(\alpha\) for all null parameter values cannot have power exceeding \(\alpha\) against this alternative. This is why an exact \(t\) test for \(\beta_1\) alone is not defined by the model, even though a test for \(\beta_1+\beta_2\) is.

Common Pitfall

Wrong reasoning: “Software returned a least-squares coefficient vector, so I can test any coefficient or contrast in that vector.” A computation routine can select one solution among many, but that selection does not make a nonestimable coefficient identifiable. Another least-squares solution can give a different value for that coefficient while leaving every fitted value unchanged.

Correction: Check the contrast itself before forming a standard error or test statistic. Verify that \(c\) is in the row space of \(X\), or equivalently that \(c'v=0\) for every null-space vector \(v\). If it is estimable, its value is invariant across least-squares solutions and the exact \(t\) result applies under the stated Gaussian assumptions. If it is not, changing parameter values along a null-space direction changes the contrast without changing the distribution of the observations.

Key takeaway: Rank deficiency does not make all inference impossible. It makes some coefficient combinations unidentified. Exact contrast \(t\) inference is available for nonzero estimable contrasts because their values are determined by the model mean and have a linear unbiased estimator; a nonestimable contrast has neither property.

Check Your Understanding

Use the row-space or null-space conditions to answer each question.

If \(X\) has null space spanned by \(v=(1,-1,0)'\), is \(c'\beta\) with \(c=(1,1,0)'\) estimable? Is the contrast with \(c=(1,0,0)'\) estimable?
Why can two different least-squares coefficient vectors produce the same fitted values in a rank-deficient design?
For \(X\) in the example with third column equal to the sum of the first two, which combinations of coefficients are directly represented by the model mean?
What assumptions are needed for the exact \(t\) calibration in this tutorial, and what are its residual degrees of freedom?
Why can a nonestimable contrast take different values at parameter vectors that yield the same distribution of the observations?