Restricted and Unrestricted GLM Hypotheses

Null Hypotheses as Parameter Restrictions

A GLM hypothesis test compares a specified null model with a larger model. The null hypothesis says that the parameter lies in a subset of the full parameter space: for example, one or more regression coefficients may be zero, or a linear combination of coefficients may equal a specified value. The key count is not the total number of coefficients displayed in the model. It is the number of independent restrictions imposed by the null hypothesis.

Let \(\theta\) denote the full parameter vector, which may contain regression coefficients and, when estimated, a dispersion parameter. Write \(\Theta\) for its parameter space and \(L(\theta)\) for the likelihood. A null hypothesis can be expressed as a subset \(\Theta_0\) of \(\Theta\). The unrestricted maximum-likelihood estimate maximizes the likelihood over \(\Theta\); the restricted maximum-likelihood estimate maximizes it over \(\Theta_0\).

Definition: For a null hypothesis with parameter set \(\Theta_0\subseteq\Theta\), the unrestricted MLE \(\hat{\theta}\) maximizes \(L(\theta)\) over \(\Theta\). The restricted MLE \(\hat{\theta}_0\) maximizes \(L(\theta)\) over \(\Theta_0\). If the null is written \(h(\theta)=0\), then \(\Theta_0=\{\theta\in\Theta:h(\theta)=0\}\). The two estimates are generally different because they maximize over different sets.

For a GLM with coefficient vector \(\beta\), common null hypotheses are linear restrictions such as \(A\beta=c\), where \(A\) is a known matrix and \(c\) a known vector. If the full parameter vector also includes an unknown dispersion parameter, a hypothesis about regression coefficients alone leaves that parameter unrestricted. It is then a nuisance parameter under the null, not an additional restriction.

Counting Independent Linear Restrictions

Suppose \(\theta\in\mathbb{R}^p\) and the null is \(A\theta=c\), with \(A\) having \(m\) rows. The number of restrictions is the row rank of \(A\), not necessarily the number \(m\) of equations written down. A row that is a linear combination of other rows contributes no new restriction. The system must also be consistent: if no \(\theta\) satisfies it, it does not define a possible null model.

Theorem: If \(A\theta=c\) is consistent and \(\operatorname{rank}(A)=q\), then its solution set in \(\mathbb{R}^p\) is an affine space of dimension \(p-q\). Thus the null imposes \(q\) independent restrictions. In a regular, interior GLM problem, this is the restriction count used for the usual asymptotic chi-square degrees of freedom of a test comparing the full and null models.

Proof. Choose one solution \(\theta_0\), which exists by consistency. Every solution can be written \(\theta_0+v\), where \(Av=0\), because \(A(\theta_0+v)=c\) exactly when \(Av=0\). Conversely, every vector of this form is a solution. By the rank-nullity theorem, the null space of \(A\) has dimension \(p-\operatorname{rank}(A)=p-q\). Translation by \(\theta_0\) does not change dimension, so the solution set has dimension \(p-q\). In a regular interior model, restricting from \(p\) free coordinates to \(p-q\) leaves \(q\) fewer local parameter dimensions, giving the stated restriction count. \(\square\)

The theorem counts independent restrictions on the parameter, not the number of predictors in the model, the sample size, or the number of observations used to estimate coefficients. If a dispersion parameter is included in \(\theta\) and is left free under the null, it contributes to both model dimensions and does not increase the difference.

Worked Example: Testing Two Logistic-Regression Slopes

Consider a logistic regression with \(\beta=(\beta_0,\beta_1,\beta_2,\beta_3)^\mathsf{T}\), where \(\beta_0\) is the intercept. Suppose the null hypothesis is that predictors 2 and 3 have no effects: \(\beta_2=\beta_3=0\). In matrix form,

A\beta=0,\qquad A= \begin{pmatrix} 0&0&1&0\\ 0&0&0&1 \end{pmatrix}.

The two rows of \(A\) are linearly independent, so \(\operatorname{rank}(A)=2\). The full coefficient vector has dimension \(p=4\); the null leaves \(\beta_0\) and \(\beta_1\) free, so its dimension is \(4-2=2\). The null therefore imposes two independent restrictions. If an unknown dispersion parameter were also estimated and left free under both models, it would not change this difference.

The restricted fit is a logistic regression containing the intercept and predictor 1, with the coefficients of predictors 2 and 3 fixed at zero. The unrestricted fit estimates all four coefficients. The restricted estimate is not obtained by taking the unrestricted estimates and replacing their last two entries by zero: the remaining coefficients must be re-estimated under the restriction.

Restricted and Unrestricted Fits in Practice

The distinction between the estimates follows directly from the optimization domains. The restricted estimate is allowed to vary only within the null set; the unrestricted estimate can vary throughout the full parameter space. In general, the restricted fit adjusts all free parameters to maximize the likelihood subject to the null, including nuisance parameters that remain free.

Theorem: If \(\Theta_0\subseteq\Theta\), and both maximum-likelihood estimates exist, then \(L(\hat{\theta}_0)\leq L(\hat{\theta})\), or equivalently \(\ell(\hat{\theta}_0)\leq\ell(\hat{\theta})\), where \(\ell\) is the log-likelihood.

Proof. By definition, \(\hat{\theta}\) maximizes \(L\) over \(\Theta\). Since \(\hat{\theta}_0\in\Theta_0\subseteq\Theta\), the value \(L(\hat{\theta}_0)\) is one of the values considered in the maximization over \(\Theta\), and therefore cannot exceed \(L(\hat{\theta})\). The logarithm is increasing on positive numbers, so the same inequality holds for the log-likelihood. \(\square\)

Worked Example: Restricted and Unrestricted Logistic MLEs

Suppose eight independent Bernoulli outcomes are observed: at \(x=-1\), there is 1 success and 3 failures; at \(x=1\), there are 3 successes and 1 failure. Fit the logistic model \(\operatorname{logit}\{P(Y=1\mid x)\}=\beta_0+\beta_1x\). Compare the unrestricted model with the null hypothesis \(\beta_1=0\).

The unrestricted model has separate fitted probabilities at the two observed predictor values. The maximum-likelihood estimate at each value is its observed success fraction: \(\hat p_{-1}=1/4\) and \(\hat p_1=3/4\). Their logits are \(-\log 3\) and \(\log 3\), respectively. Solving \(\beta_0-\beta_1=-\log 3\) and \(\beta_0+\beta_1=\log 3\) gives \(\hat\beta_0=0\) and \(\hat\beta_1=\log 3\).

Under the restriction \(\beta_1=0\), all eight outcomes have the same fitted probability. The likelihood is proportional to \(p^4(1-p)^4\), which is maximized at \(\hat p=4/8=1/2\). Hence the restricted estimate is \(\hat\beta_{0,0}=0\), with \(\hat\beta_{1,0}=0\).

Ignoring parameter-free constants, the unrestricted likelihood and log-likelihood are

L(\hat\beta)=(1/4)^2(3/4)^6,\qquad \ell(\hat\beta)=2\log(1/4)+6\log(3/4)\approx -4.499.

The restricted likelihood and log-likelihood are

L(\hat\beta_0)=(1/2)^8,\qquad \ell(\hat\beta_0)=8\log(1/2)\approx -5.545.

As expected, \(-5.545<-4.499\). The full coefficient vector has dimension 2, and the null leaves only the intercept free. There is one independent restriction, so the relevant restriction count is 1.

Nonlinear Restrictions and Regularity

Not every null is linear. It may be written \(h(\theta)=0\), with \(h\) taking values in \(\mathbb{R}^m\). At a point satisfying the null, form the Jacobian matrix \(D h(\theta)\), whose rows are the gradients of the component restrictions. If this matrix has constant full row rank \(q\) locally, the implicit function theorem gives a smooth null set of local dimension \(p-q\). Thus the local number of independent restrictions is the Jacobian rank, rather than simply the number of equations written.

Key distinction: A regular null with \(q\) independent restrictions has \(q\) fewer local parameter dimensions than the full model. This dimension count supplies the usual asymptotic degrees of freedom only under the regularity conditions for the test. Boundary points, singular restrictions, nonidentifiability, or failure of the MLE to exist can invalidate the ordinary chi-square count.

Worked Example: Redundant Restrictions in a Poisson GLM

Let \(Y_i\) be independent Poisson responses with \(\log E[Y_i\mid x_i]=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}\). Consider the null written as three equations: \(\beta_1+\beta_2=0\), \(2\beta_1+2\beta_2=0\), and \(\beta_0=0\). The coefficient vector has dimension \(p=3\), and the restriction matrix is

A= \begin{pmatrix} 0&1&1\\ 0&2&2\\ 1&0&0 \end{pmatrix},\qquad c=0.

The second row is twice the first, so it adds no independent condition. The first and third rows are independent, giving \(\operatorname{rank}(A)=2\), not 3. Solving the restrictions gives \(\beta_0=0\) and \(\beta_2=-\beta_1\), with \(\beta_1\) free. The null therefore has dimension \(3-2=1\), and the restriction count is 2.

The restricted Poisson fit must estimate the remaining free coefficient under these conditions. Counting the three written equations as three restrictions would incorrectly conclude that no coefficient is free.

Common Mistakes and Why the Count Matters

Counting equations instead of independent restrictions. Use the rank of the restriction matrix for linear hypotheses. Redundant rows do not reduce the parameter dimension again.
Changing the unrestricted estimate to satisfy the null. The restricted MLE is obtained by maximizing over the null set; all parameters still free under that null must be re-estimated.
Counting every parameter in the model as a restriction. Parameters left free under the null, including nuisance parameters, do not contribute to the dimension difference.
Assuming every nonlinear equation contributes one regular restriction. At singular points, the Jacobian can lose rank; the ordinary dimension and degrees-of-freedom argument may not apply.

Getting the null set and its dimension right is essential before calculating any test statistic. It determines which fit is restricted, which is unrestricted, and—when the regular asymptotic theory applies—the number of independent restrictions used for the reference degrees of freedom.

Check Your Understanding

For each question, identify the null parameter set or the independent restriction count.

A GLM has five regression coefficients and tests whether the last two are both zero. How many independent restrictions are imposed, assuming they are distinct coefficients?
If a consistent linear restriction matrix has four rows but rank three, how many independent restrictions does it impose?
Why must the remaining free coefficients generally be re-estimated when finding a restricted MLE?
A model includes an unknown dispersion parameter that is left unrestricted under a null hypothesis about slopes. Does it add to the restriction count? Explain.
What regularity feature of a nonlinear restriction determines its local number of independent restrictions?