Why Restriction Rank Determines the Numerator Degrees of Freedom
A general linear hypothesis may be written as several equations, but the equations need not impose several distinct requirements. For example, one equation might be a linear combination of two others. The numerator degrees of freedom in the exact F test count the independent restrictions, not the number of lines used to write them.
The reason is distributional: the numerator is a quadratic form in the estimated departures from the null restrictions. Its number of independent normal components is the rank of the restriction matrix. We derive that connection under the fixed-design normal linear model.
Because \(\mathbf X\) has full column rank, the least-squares estimator is \(\hat{\boldsymbol\beta}=(\mathbf X^\top\mathbf X)^{-1}\mathbf X^\top\mathbf y\), and \(\operatorname{Var}(\hat{\boldsymbol\beta})=\sigma^2(\mathbf X^\top\mathbf X)^{-1}\). For \(r\) independent restrictions, we can choose \(r\) independent rows of \(\mathbf C\) and use the corresponding entries of \(\mathbf d\). Write the resulting matrix and vector as \(\mathbf B\) and \(\boldsymbol\delta\). Then \(\mathbf B\) has row rank \(r\), and the same null hypothesis can be written \(H_0:\mathbf B\boldsymbol\beta=\boldsymbol\delta\).
Redundant Equations and Independent Restrictions
If the rows of \(\mathbf C\) are dependent, the covariance matrix of \(\mathbf C\hat{\boldsymbol\beta}\) is singular. In that case, the usual inverse-based quadratic form cannot be computed using all \(q\) equations. Instead, reduce the restrictions to an independent equivalent set.
To verify the equivalence, if \(\mathbf B\boldsymbol\beta=\boldsymbol\delta\), multiplication by \(\mathbf K\) gives \(\mathbf C\boldsymbol\beta=\mathbf d\). Conversely, if \(\mathbf C\boldsymbol\beta=\mathbf d\), then \(\mathbf K(\mathbf B\boldsymbol\beta-\boldsymbol\delta)=\mathbf 0\). Since \(\mathbf K\) has full column rank, its null space contains only the zero vector, so \(\mathbf B\boldsymbol\beta-\boldsymbol\delta=\mathbf 0\). Thus the reduced system imposes precisely the same null hypothesis. The reduction also explains why an incompatible target \(\mathbf d\) cannot be treated as a redundant version of the same hypothesis: it may make the original equations impossible to satisfy.
Worked Example: Three Written Equations, Two Restrictions
Consider the restrictions \(\beta_1+\beta_3=2\), \(\beta_2-\beta_3=-1\), and \(\beta_1+\beta_2=1\). Their matrix form is \[ \mathbf C= \begin{pmatrix} 1&0&1\\ 0&1&-1\\ 1&1&0 \end{pmatrix}, \qquad \mathbf d= \begin{pmatrix} 2\\-1\\1 \end{pmatrix}. \]
The third row of \(\mathbf C\) is the sum of the first two, and \(1=2+(-1)\) for the corresponding entries of \(\mathbf d\). Thus the third equation follows from the first two. The first two rows are linearly independent, so \(\operatorname{rank}(\mathbf C)=2\). An equivalent independent system is formed by the first two equations, and the numerator degrees of freedom are \(2\), not \(3\).
Worked Example: Detecting Redundancy by Row Reduction
Let \(\mathbf C=\begin{pmatrix}1&1&0\\0&1&1\\1&2&1\end{pmatrix}\) and \(\mathbf d=(0,1,1)^\top\). The third row of \(\mathbf C\) equals the first row plus the second row, and the third component of \(\mathbf d\) is also the sum of the first two. Therefore the third restriction is implied by the first two.
The first two rows are not multiples of one another, so they are independent. Hence the rank is \(2\), and the hypothesis has two independent restrictions. For instance, the first two equations require \(\beta_1+\beta_2=0\) and \(\beta_2+\beta_3=1\); adding them gives \(\beta_1+2\beta_2+\beta_3=1\), exactly the third equation. The numerator degrees of freedom are therefore \(2\).
Deriving the Numerator Degrees of Freedom
For an independent restriction matrix \(\mathbf B\), the estimated restriction vector \(\mathbf B\hat{\boldsymbol\beta}-\boldsymbol\delta\) has \(r\) coordinates. Under the null, its mean is zero, and its covariance is \(\sigma^2\mathbf B(\mathbf X^\top\mathbf X)^{-1}\mathbf B^\top\). The matrix between the factors of \(\sigma^2\) is positive definite: for any nonzero \(r\)-vector \(\mathbf a\), \(\mathbf a^\top\mathbf B(\mathbf X^\top\mathbf X)^{-1}\mathbf B^\top\mathbf a>0\), because \(\mathbf B^\top\mathbf a\ne\mathbf 0\) and \((\mathbf X^\top\mathbf X)^{-1}\) is positive definite.
Proof. Under \(H_0\), \(\mathbf B\boldsymbol\beta=\boldsymbol\delta\), so \(\mathbf B\hat{\boldsymbol\beta}-\boldsymbol\delta=\mathbf B(\hat{\boldsymbol\beta}-\boldsymbol\beta)\). Since \(\hat{\boldsymbol\beta}-\boldsymbol\beta=(\mathbf X^\top\mathbf X)^{-1}\mathbf X^\top\boldsymbol\varepsilon\), this is a normal vector with mean zero and covariance \(\sigma^2\mathbf A\). The matrix \(\mathbf A\) is positive definite, so it has a symmetric inverse square root. Consequently \[ \mathbf Z=\sigma^{-1}\mathbf A^{-1/2} (\mathbf B\hat{\boldsymbol\beta}-\boldsymbol\delta) \sim N_r(\mathbf 0,\mathbf I_r). \] It follows that \(Q/\sigma^2=\mathbf Z^\top\mathbf Z\), the sum of squares of \(r\) independent standard normal variables. By definition, this has a \(\chi^2_r\) distribution.
For the denominator, let \(\mathbf P_X=\mathbf X(\mathbf X^\top\mathbf X)^{-1}\mathbf X^\top\) and \(\mathbf M_X=\mathbf I_n-\mathbf P_X\). The residual vector is \(\mathbf M_X\boldsymbol\varepsilon\), so \(\operatorname{SSE}=\boldsymbol\varepsilon^\top\mathbf M_X\boldsymbol\varepsilon\). The matrix \(\mathbf M_X\) is an orthogonal projection of rank \(n-p\). Expressing the normal error vector in an orthonormal basis of its range shows that \(\operatorname{SSE}/\sigma^2\) is a sum of squares of \(n-p\) independent standard normal variables; hence it has a \(\chi^2_{n-p}\) distribution.
The numerator depends on the projection of \(\boldsymbol\varepsilon\) onto the column space of \(\mathbf X\), while the residual depends on its projection onto the orthogonal complement. These projections have zero covariance and are jointly normal, so they are independent. Thus \(Q/\sigma^2\) and \(\operatorname{SSE}/\sigma^2\) are independent. Dividing each by its degrees of freedom gives independent chi-square variables divided by their respective degrees of freedom, whose ratio has the \(F_{r,n-p}\) distribution. This proves the result. \(\square\)
Worked Example: Two Independent Restrictions in a Four-Parameter Model
Suppose \(n=20\), \(p=4\), and the hypothesis is \(\beta_1+\beta_2=0\) and \(\beta_3-\beta_4=2\). Its matrix is \[ \mathbf B= \begin{pmatrix} 1&1&0&0\\ 0&0&1&-1 \end{pmatrix}, \qquad \boldsymbol\delta= \begin{pmatrix}0\\2\end{pmatrix}. \] The two rows are independent: the first has nonzero entries only in columns 1 and 2, while the second has nonzero entries only in columns 3 and 4. Therefore \(r=2\).
The residual degrees of freedom are \(n-p=20-4=16\). Under the fixed-design normal model and the null, the test statistic has distribution \(F_{2,16}\). The numerator has two degrees of freedom because there are two independent coordinates in the restriction vector; it does not have four degrees of freedom because the parameter vector has four coordinates.
What the Rank Counts—and a Common Mistake
The rank is the dimension of the set of independent linear combinations of the restrictions. It is not necessarily the number \(q\) of rows in the matrix, nor the number \(p\) of model parameters. Once the redundant equations are removed, the standardized restriction error has \(r\) independent standard normal components. Squaring and adding those components gives the numerator chi-square with \(r\) degrees of freedom; dividing by \(r\) is what produces the numerator mean square in the F statistic.
A common mistake is to count every written equation as one numerator degree of freedom. That reasoning fails when some equations are linear combinations of others. For example, three equations with the third equal to the sum of the first two do not provide three independent pieces of information. The correction is to find \(\operatorname{rank}(\mathbf C)\), check that the target values have the same dependencies, and formulate the quadratic statistic using an independent set of restrictions.
The exact result also depends on its assumptions. In particular, the Gaussian spherical error model gives the exact normal and chi-square distributions, and the full-column-rank design gives the stated least-squares covariance. Rank alone identifies the numerator degrees of freedom; it does not, by itself, guarantee an exact F calibration if those distributional assumptions do not hold.
Check Your Understanding
Use the rank of the restriction matrix to answer each question.
- A hypothesis is written with four equations, but one equation is the sum of two others and all target values obey the same relation. What determines its numerator degrees of freedom?
- Why does \(\mathbf B(\mathbf X^\top\mathbf X)^{-1}\mathbf B^\top\) have an inverse when \(\mathbf B\) has independent rows and \(\mathbf X\) has full column rank?
- In the fixed-design normal model, what distribution does the scaled numerator quadratic form have under the null?
- For \(n=30\), \(p=5\), and a hypothesis of rank \(3\), what are the numerator and denominator degrees of freedom?
- Why is counting rows of a restriction matrix not always a valid way to find numerator degrees of freedom?