Degrees of Freedom from the Rank of Restrictions

Why the Number of Restrictions Can Be Misleading

A multivariate hypothesis may be written as several equations, but the number of equations is not necessarily the number of independent restrictions. Some equations may repeat others or be linear combinations of them. For a regular Wald test, the relevant count is the rank of the restriction Jacobian at the null parameter. That rank determines the dimension of the first-order variation being tested and, under suitable conditions, the chi-square degrees of freedom.

There is an important qualification: a Jacobian describes local, first-order changes. If its rank falls at the null while the restrictions still impose additional constraints through higher-order terms, the rank alone does not describe the full local null set. The distinction between exact redundancy and such local dependence is central to using the degrees-of-freedom rule correctly.

Restrictions and Their Jacobian

Definition: Let $\theta\in\mathbb{R}^p$ be a parameter and let $g:\Theta\to\mathbb{R}^r$ be a differentiable restriction function, with components $g_1,\ldots,g_r$. The null hypothesis is $H_0:g(\theta)=0$, meaning that all $r$ components equal zero. The restriction Jacobian at $\theta_0$ is the $r\times p$ matrix $$ G(\theta_0)=\left.\frac{\partial g(\theta)}{\partial\theta^{\mathsf T}}\right|_{\theta=\theta_0}. $$ Its rank is the number of linearly independent rows, equivalently the number of locally independent first-order restriction directions at $\theta_0$.

The phrase “first-order” matters. The rank counts independent gradients, not necessarily the number of equations and not automatically the number of dimensions removed by the exact nonlinear null set. For a regular restriction system, these counts agree locally; in a singular system, they can differ.

Suppose an estimator $\hat\theta$ satisfies $\sqrt n(\hat\theta-\theta_0)\xrightarrow{d}N(0,V)$ under $H_0$, where $V$ is positive definite, and suppose $g$ is differentiable near $\theta_0$. When $G(\theta_0)$ has full row rank $r$, the familiar Wald statistic is

W_n=n\,g(\hat\theta)^{\mathsf T} \left\{G(\hat\theta)\hat V G(\hat\theta)^{\mathsf T}\right\}^{-1} g(\hat\theta),

where $\hat V$ consistently estimates $V$. The inverse exists asymptotically when the restriction covariance has full row rank. If the rows of $G(\theta_0)$ are dependent, that covariance is singular, so the ordinary inverse formula is not available.

The Rank Rule for a Regular Wald Test

Theorem: Suppose $\theta_0$ is an interior point of $\Theta$, $g(\theta_0)=0$, $g$ is continuously differentiable near $\theta_0$, and $\sqrt n(\hat\theta-\theta_0)\xrightarrow{d}N(0,V)$ with $V$ positive definite. Let $G_0=G(\theta_0)$ have rank $q$. If the rank of $G(\theta)$ is constantly $q$ in a neighborhood of $\theta_0$, then the first-order Wald statistic using the full restriction vector and the Moore–Penrose inverse has an asymptotic $\chi^2_q$ distribution under $H_0$, provided the covariance estimates are consistent and the estimated rank is $q$ with probability tending to one.

The Moore–Penrose inverse $A^+$ of a symmetric nonnegative matrix $A$ acts as the ordinary inverse on its nonzero eigenspaces and as zero on its null space. In this theorem it allows the full vector of restrictions to be retained when some rows are redundant.

To see why the degrees of freedom are $q$, write $G_0=G(\theta_0)$ and $A=G_0VG_0^{\mathsf T}$. Differentiability gives the expansion

\sqrt n\,g(\hat\theta) =G_0\sqrt n(\hat\theta-\theta_0)+o_p(1) \xrightarrow{d} Z,\qquad Z\sim N(0,A).

Because $V$ is positive definite, $\operatorname{rank}(A)=\operatorname{rank}(G_0)=q$. For the limiting quadratic form, diagonalize $A$ as $A=U\operatorname{diag}(\lambda_1,\ldots,\lambda_q,0,\ldots,0)U^{\mathsf T}$, with each $\lambda_j>0$ and $U$ orthogonal. The first $q$ coordinates of $U^{\mathsf T}Z$ are independent normal variables with variances $\lambda_1,\ldots,\lambda_q$; the remaining coordinates are zero with probability one. Therefore

Z^{\mathsf T}A^+Z =\sum_{j=1}^{q}\frac{(U^{\mathsf T}Z)_j^2}{\lambda_j} \sim\chi^2_q.

Consistency of the covariance and Jacobian estimates, together with the constant-rank condition, gives the same limit for the estimated Wald statistic. The constant-rank condition is not a cosmetic technicality: it excludes cases in which the derivative loses rank exactly at the null even though the restrictions impose additional constraints through higher-order terms. $\square$

When the Jacobian has full row rank, $q=r$, so the usual inverse Wald statistic has $r$ degrees of freedom. When some restrictions are exactly redundant and the rank is $q<r$, the generalized-inverse statistic has $q$ degrees of freedom under the theorem’s conditions.

Worked Example: Two Independent Restrictions Written Three Times

Let $\theta=(\theta_1,\theta_2,\theta_3)^{\mathsf T}$, and test $\theta_1=\theta_2$ and $\theta_3=0$ using the three-component restriction $g(\theta)=(\theta_1-\theta_2,\;2\theta_1-2\theta_2,\;\theta_3)^{\mathsf T}$. The second component repeats the first up to a factor of two.

The Jacobian is

G= \begin{pmatrix} 1&-1&0\\ 2&-2&0\\ 0&0&1 \end{pmatrix}.

The second row is twice the first, while the third row is independent of the first. Thus $\operatorname{rank}(G)=2$, not 3. Suppose $V=I_3$, $n=100$, and the estimated parameter has $\hat\theta_1-\hat\theta_2=0.1$ and $\hat\theta_3=0.2$. The restriction vector is $(0.1,0.2,0.2)^{\mathsf T}$, and its covariance factor is

GVG^{\mathsf T}= \begin{pmatrix} 2&4&0\\ 4&8&0\\ 0&0&1 \end{pmatrix}.

The first two coordinates contain only one direction of variation. Their contribution to the generalized-inverse quadratic form is $0.1^2/2=0.005$; the third contributes $0.2^2=0.04$. Hence

$$ W_n=100(0.005+0.04)=4.5. $$

The regular reference distribution is $\chi^2_2$, because there are two independent restriction directions. Counting three written equations would give the wrong degrees of freedom.

When Redundant Equations Can Be Reduced

It is often simpler to remove redundant equations and use a smaller restriction vector with a nonsingular covariance matrix. This is valid only when the reduced system has the same null set locally as the original system. Independent gradients alone are not enough: dropping equations can change the hypothesis even if the retained gradients are independent.

For exact linear redundancies, this check is usually straightforward. If the full restriction vector is $g(\theta)=B h(\theta)$, where $B$ has full column rank, then $g(\theta)=0$ if and only if $h(\theta)=0$. In that case $h$ is a locally equivalent reduced system. More generally, reduction is justified when the two zero sets agree in a neighborhood of the null parameter. If equivalence has not been established, retaining the full vector avoids silently changing the null hypothesis, though a first-order Wald test may still fail to capture higher-order constraints.

Worked Example: A Nonlinear Restriction With the Same Local Null Set

Consider $\theta=(\theta_1,\theta_2)^{\mathsf T}$, the null point $\theta_0=(1,0)^{\mathsf T}$, and the two-component restriction $g(\theta)=(\theta_1^2-1,\theta_2)^{\mathsf T}$. Near $\theta_0$, the first equation is equivalent to $\theta_1=1$, since $\theta_1^2-1=(\theta_1-1)(\theta_1+1)$ and $\theta_1+1$ is nonzero in a sufficiently small neighborhood of 1. Thus the local null set is exactly $\theta_1=1,\theta_2=0$.

At the null, the Jacobian and its rank are

G(\theta_0)= \begin{pmatrix} 2&0\\ 0&1 \end{pmatrix}, \qquad \operatorname{rank}\{G(\theta_0)\}=2.

Suppose $V=\operatorname{diag}(4,1)$, $n=25$, and $\hat\theta=(1.1,0.2)^{\mathsf T}$. Using the null Jacobian for the first-order Wald calculation, the covariance factor is $G(\theta_0)VG(\theta_0)^{\mathsf T}=\operatorname{diag}(16,1)$. The restriction values are $g(\hat\theta)=(0.21,0.2)^{\mathsf T}$, since $1.1^2-1=0.21$. Therefore

W_n=25\left(\frac{0.21^2}{16}+0.2^2\right) =25(0.00275625+0.04)=1.06890625.

There are two independent local restriction directions, so the regular asymptotic reference distribution is $\chi^2_2$. The nonlinear form of the first equation does not change the degrees of freedom here because its gradient is nonzero and its zero set is locally equivalent to a single smooth restriction.

Local Dependence and the Limits of the Rank Rule

A restriction system can have dependent gradients at the null without having redundant equations. In that case, the Jacobian gives only the first-order picture. The constant-rank condition in the theorem fails, and assigning chi-square degrees of freedom from the rank alone can give a misleading description of the full hypothesis.

Worked Example: Rank One but a Two-Constraint Null Set

Let $\theta=(x,y)^{\mathsf T}$, let $\theta_0=(0,0)^{\mathsf T}$, and define $g(x,y)=(x,\;x+y^2)^{\mathsf T}$. The joint null requires $x=0$ and $x+y^2=0$. Substituting the first equation into the second gives $y^2=0$, so the exact null set consists only of $(0,0)$.

However, at the null the Jacobian is

G(0,0)= \begin{pmatrix} 1&0\\ 1&0 \end{pmatrix}, \qquad \operatorname{rank}\{G(0,0)\}=1.

The two gradients are identical, but the second equation still imposes a condition on $y$ through its quadratic term. Away from $y=0$, the Jacobian is $\begin{pmatrix}1&0\\1&2y\end{pmatrix}$, which has rank 2. Thus its rank is not constant near the null.

For an estimator with a root-$n$ error in both coordinates, the first-order expansion of $\sqrt n\,g(\hat\theta)$ sees only the $x$-direction: the contribution of $\hat y^2$ is of smaller order under the null. A first-order generalized-inverse Wald calculation therefore reflects rank one and does not account for the quadratic restriction on $y$. In particular, the rank-one calculation should not be described as a regular two-dimensional test of the exact point null. A different analysis is needed to capture the higher-order constraint.

This example also shows why selecting a subset of restrictions merely because its gradients are independent can be invalid. Keeping only $g_1(x,y)=x$ changes the null set from the single point $(0,0)$ to the entire line $x=0$. A reduced system is legitimate only when its zero set is locally equivalent to the original one.

Practical Checks and Common Pitfalls

Before assigning degrees of freedom, evaluate the Jacobian at the null parameter and check both its rank and the local behavior of the restriction system. In a regular problem, the rank gives the chi-square degrees of freedom. In a singular problem, it counts only first-order directions, and the usual calibration may not describe the exact null.

Counting equations instead of directions. Three equations can have rank two. Use the Jacobian rank, not the number of rows.
Dropping equations based only on gradients. Independent gradients in a retained subset do not guarantee that the discarded equations are implied. Verify local equality of the zero sets.
Treating every rank deficiency as harmless redundancy. Exact duplicates are different from restrictions whose gradients coincide at the null but whose higher-order terms impose additional conditions.
Using an ordinary inverse for a singular covariance. If the full restriction covariance is singular, the usual inverse does not exist. A generalized inverse gives the regular rank-based quadratic form only under appropriate conditions, including stable rank.

Key takeaway: For a regular Wald test, the chi-square degrees of freedom equal the rank of the restriction Jacobian at the null. When rank is deficient, use a reduced system only if it preserves the local null set; when rank changes or higher-order restrictions matter, the rank-based first-order calibration may be inadequate.

Check Your Understanding

Use the restriction Jacobian and the local null set—not just the number of equations—to answer these questions.

If a four-row restriction Jacobian has rank three at a regular null, what are the asymptotic chi-square degrees of freedom?
Why is the ordinary inverse in the Wald statistic unavailable when two restriction gradients are exact multiples of each other?
What must be checked before replacing a full restriction system with a smaller set of restrictions?
For $g(x,y)=(x,x+y^2)^{\mathsf T}$ at $(0,0)$, what is the Jacobian rank, and what is the exact joint null set?
Why does a rank calculation alone fail to capture every feature of a singular restriction system?