Why Degrees of Freedom Depend on the Restrictions
A chi-square test statistic is not interpreted by its numerical value alone: its reference distribution also depends on how many independent aspects of the null hypothesis are being tested. In a generalized linear model (GLM), a hypothesis may be written using several equations, but some equations can repeat information already contained in others. The relevant number is therefore the rank of the restrictions, not necessarily the number of equations written down.
This count applies to Wald, score, and likelihood-ratio tests under regular large-sample conditions. For each, the degrees of freedom reflect the dimension by which the null model is smaller than the unrestricted model. The test statistics need not have the same observed value, but their usual asymptotic chi-square distributions use this same dimension count.
Linear Restrictions and Their Rank
Consistency matters: if the equations contradict one another, there is no parameter value satisfying the null hypothesis. For consistent equations, each independent restriction removes one dimension from the coefficient space. If the GLM also contains nuisance parameters that are unrestricted under the null, those parameters remain in both models and do not change this dimension difference.
The dimension statement follows directly from linear algebra. Choose any solution \(\beta_0\) to \(R\beta=r\). Every other solution is \(\beta_0+v\), where \(Rv=0\). The allowable directions \(v\) form the null space of \(R\), whose dimension is \(p-\operatorname{rank}(R)\) by the rank-nullity theorem.
Redundant equations do not change the null set when they are consistent with the independent equations. For instance, \(\beta_1=0\), \(\beta_2=0\), and \(\beta_1+\beta_2=0\) are three written equations but only two independent restrictions. Counting rows would give the wrong degrees of freedom.
Why the Wald Test Uses the Rank
The rank count can be seen directly from the Wald statistic. Reduce the restrictions to \(A\beta=a\), where \(A\) has \(q\) linearly independent rows and the reduced equations describe the same null set. Let \(\hat\beta\) be the unrestricted maximum likelihood estimator and let \(\widehat V\) estimate its covariance matrix. The Wald statistic for these restrictions is
Here the inverse exists when the estimated covariance is positive definite and \(A\) has full row rank. The following result gives the asymptotic degrees of freedom.
Proof. Under the null, \(A\hat\beta-a=A(\hat\beta-\beta_0)\). Define \(Z_n=\sqrt{n}(\hat\beta-\beta_0)\). By the assumed asymptotic normality, \(AZ_n\xrightarrow{d}N(0,A\Sigma A^\mathsf{T})\). The matrix \(A\Sigma A^\mathsf{T}\) is positive definite: for any nonzero \(u\in\mathbb{R}^q\), full row rank of \(A\) implies \(A^\mathsf{T}u\ne0\), and therefore
Since \(n\widehat V\xrightarrow{p}\Sigma\), Slutsky’s theorem gives
Whitening \(Z\) by the inverse square root of its positive definite covariance produces a \(q\)-dimensional standard normal vector. Its squared Euclidean length has a \(\chi^2_q\) distribution. Thus the degrees of freedom are \(q\), the number of independent restrictions. \(\square\)
This result assumes a regular interior parameter point and the stated asymptotic normality and covariance consistency. It is not a general guarantee for boundary hypotheses, nonidentifiable models, or other nonregular settings.
Degrees of Freedom Across the Three Tests
For a regular, nested GLM comparison, the likelihood-ratio statistic is twice the difference between the unrestricted and restricted maximized log-likelihoods. Under the null and the regularity conditions of Wilks’ theorem, it converges to a chi-square distribution with degrees of freedom equal to the difference in model dimensions. For linear restrictions of rank \(q\), that difference is \(q\): the unrestricted coefficient vector has dimension \(p\), while the restricted one has dimension \(p-q\). Unrestricted nuisance parameters appearing in both models cancel in the difference.
The score test has the same degrees-of-freedom count when it tests a regular, identifiable \(q\)-dimensional restriction and the information for that tested subspace, after accounting for nuisance parameters, is nonsingular. Redundant coordinates do not create additional independent score directions. These are large-sample reference distributions; the three statistics can differ in finite samples.
Worked Applications
Worked Example: One Logistic Regression Coefficient
Consider independent Bernoulli responses with a logistic mean model, \(\operatorname{logit}(\pi_i)=\beta_0+\beta_1x_i\). Test \(H_0:\beta_1=0\). The restriction matrix is \(R=(0\ \ 1)\), so its rank is \(q=1\). Suppose the unrestricted estimate is \(\hat\beta_1=0.40\), with estimated standard error \(0.20\). The Wald statistic is
Its usual large-sample reference distribution is \(\chi^2_1\), because the null imposes one independent restriction. The corresponding two-sided large-sample \(p\)-value is approximately \(0.0455\). A score or likelihood-ratio test of this same one-dimensional null also uses one degree of freedom under regularity, though its statistic and resulting \(p\)-value need not equal the Wald values.
Worked Example: Three Equations but Only Two Restrictions
Suppose a logistic model has coefficient vector \(\beta=(\beta_0,\beta_1,\beta_2)^\mathsf{T}\), and the null is written as
The restriction matrix has rows \((0,1,0)\), \((0,0,1)\), and \((0,1,1)\). The third row is the sum of the first two, so the matrix has rank \(q=2\). The third equation adds no new condition: whenever the first two hold, the third holds as well. Thus the restricted model has dimension \(3-2=1\), not \(3-3=0\).
For a numerical Wald calculation, suppose the unrestricted estimate is \((0.1,0.30,-0.20)^\mathsf{T}\), and its estimated covariance matrix is diagonal with entries \(0.25,0.04,0.09\). Use the two independent restrictions, so the tested estimate is \((0.30,-0.20)^\mathsf{T}\), with covariance matrix \(\operatorname{diag}(0.04,0.09)\). Then
The reference distribution is \(\chi^2_2\). Trying to invert a covariance matrix formed from all three written restriction estimates would fail: the third restriction estimate is the sum of the first two, so that covariance matrix is singular. Reducing to an independent set avoids this problem and makes the effective degrees of freedom explicit.
Worked Example: A Full-Rank Joint Test in Poisson Regression
Let independent counts follow a Poisson regression with log link and coefficient vector \(\beta=(\beta_0,\beta_1,\beta_2,\beta_3)^\mathsf{T}\). The intercept is unrestricted, and the null hypothesis is that all three slopes are zero. Its restriction matrix is
The three rows are independent, so \(q=\operatorname{rank}(R)=3\). The null model has only the intercept, and the unrestricted model has an intercept and three slopes; their dimension difference is \(4-1=3\), agreeing with the rank calculation.
Suppose the estimated slopes are \((0.20,-0.30,0.40)\) and their estimated covariance matrix is diagonal with variances \(0.01,0.0225,0.04\). The Wald statistic is
This joint test uses a \(\chi^2_3\) reference distribution. The intercept contributes no tested restriction, even though it is a coefficient in the model.
Common Mistake and Practical Checks
Wrong reasoning: “There are four equations in my null hypothesis, so the test has four degrees of freedom.” This counts written equations without checking whether they provide four independent pieces of information. If one equation is a linear combination of the others, it does not reduce the model by another dimension.
Correction: Form the restriction matrix, check that the null equations are consistent, and calculate its rank. For a regular nested comparison, verify that the rank matches the dimension difference between the unrestricted and restricted models. When nuisance parameters remain unrestricted, include them in both model dimensions before taking the difference; they do not add degrees of freedom to the test.
The usual rank-based chi-square calibration also relies on regularity. Boundary parameters, nonidentifiability, or singular information can invalidate the standard result; a rank calculation alone does not repair those problems. In particular, do not apply the regular chi-square rule automatically when the null is a boundary point.
Check Your Understanding
For each question, focus on the independent restrictions and the conditions for using the usual asymptotic reference distribution.
- A coefficient vector has five components, and a consistent restriction matrix has rank two. What is the dimension of the restricted coefficient space?
- Why can a system with four written equations have only two degrees of freedom?
- A GLM has an intercept and four slopes. The null sets all four slopes to zero while leaving the intercept unrestricted. What is the rank-based degrees-of-freedom count?
- What condition on the restriction matrix ensures that the Wald covariance matrix for a reduced set of restrictions is nonsingular when the coefficient covariance is positive definite?
- Name one situation in which the usual rank-based chi-square calibration may fail even when the restriction matrix has a clear rank.