Score Tests Under Generalized Linear Model Restrictions

Why Test at the Restricted Fit?

A linear restriction on generalized linear model coefficients might assert that a slope is zero, that several coefficients vanish together, or that a specified contrast equals a known value. A score test evaluates the likelihood’s slope away from the null model, but does so at the maximum-likelihood fit that obeys the null restriction. The fit therefore uses data to estimate nuisance coefficients while keeping the tested restriction in force.

This distinction matters. The score at the unrestricted maximum-likelihood estimate is zero, so it cannot itself provide a test statistic. The relevant score is the part pointing away from the restricted fit in the tested directions, adjusted for the fact that nuisance coefficients are estimated from the same data.

Set-Up and Efficient Information

Let \(\beta\in\mathbb{R}^p\) be the coefficient vector, with log-likelihood \(\ell(\beta)\), score \(U(\beta)=\partial\ell(\beta)/\partial\beta\), and expected Fisher information \(I(\beta)=E[-\partial^2\ell(\beta)/\partial\beta\partial\beta^{\mathsf T}]\). Consider the null hypothesis \(H_0:R\beta=r\), where \(R\) is a known \(q\times p\) matrix of rank \(q\), with \(q<p\). Assume the restricted maximum-likelihood estimate exists, is an interior point of the allowed parameter space, and the information matrices used below are nonsingular.

To define that adjustment precisely, choose a \(p\times q\) matrix \(C\) such that \(RC=I_q\), and a \(p\times(p-q)\) matrix \(N\) whose columns form a basis for the null space of \(R\), so \(RN=0\). The columns of \([C\ N]\) form a coordinate system: write \(\beta=\beta_0+C\gamma+N\eta\), where \(R\beta_0=r\). Then \(R\beta-r=\gamma\), and the null hypothesis is \(\gamma=0\). The vector \(\eta\) contains nuisance coordinates.

In these coordinates, partition the expected information at the restricted estimate as \(I=\begin{pmatrix}I_{\gamma\gamma}&I_{\gamma\eta}\\I_{\eta\gamma}&I_{\eta\eta}\end{pmatrix}\). The information for \(\gamma\) after allowing \(\eta\) to vary is the Schur complement \(I_{\gamma\gamma\cdot\eta}=I_{\gamma\gamma}-I_{\gamma\eta}I_{\eta\eta}^{-1}I_{\eta\gamma}\). This is called the efficient information for the tested coordinates. At the restricted maximum, the nuisance score \(U_\eta\) is zero, but the tested score \(U_\gamma\) need not be.

A useful formula that displays the tested contrast is \(S=\{R I^{-1}U\}^{\mathsf T}\{R I^{-1}R^{\mathsf T}\}^{-1}\{R I^{-1}U\}\), where the score and information are evaluated at \(\tilde\beta\). These expressions are equivalent; they are different ways to show the same nuisance adjustment, not separate tests.

Why the Statistic Has This Form

First consider the restricted maximum. In the \((\gamma,\eta)\) coordinates, it satisfies \(\gamma=0\) and \(U_\eta(0,\tilde\eta)=0\). The information block \(I_{\gamma\gamma}\) alone treats \(\eta\) as fixed. But an observed score in the tested directions can be partly explained by changing \(\eta\). The subtraction \(I_{\gamma\eta}I_{\eta\eta}^{-1}I_{\eta\gamma}\) removes that shared component. It is the information analogue of residualizing the tested score against the nuisance score.

Here is a derivation of the coordinate formula. Let \(J=[C\ N]\) be the change-of-coordinate matrix. The information in the new coordinates is \(J^{\mathsf T}IJ\). Its inverse is \(J^{-1}I^{-1}J^{-\mathsf T}\); the upper-left \(q\times q\) block is \(RI^{-1}R^{\mathsf T}\), because the first coordinate is \(\gamma=R\beta-r\). The block-inverse identity says that this same block equals \(I_{\gamma\gamma\cdot\eta}^{-1}\). Thus \(I_{\gamma\gamma\cdot\eta}^{-1}=RI^{-1}R^{\mathsf T}\). Also \(U_\gamma=C^{\mathsf T}U\). At the restricted maximum, \(N^{\mathsf T}U=U_\eta=0\), so \(U\) lies in the row space of \(R\), or \(U=R^{\mathsf T}\lambda\) for some \(\lambda\in\mathbb{R}^q\). Consequently, the quadratic form in the full score equals the efficient-score form. In particular, \(U^{\mathsf T}I^{-1}U=(RI^{-1}U)^{\mathsf T}(RI^{-1}R^{\mathsf T})^{-1}(RI^{-1}U)\).

Asymptotic Justification

The chi-square calibration is an asymptotic result, not generally an exact finite-sample statement. For clarity, use the tested and nuisance coordinates above and suppose the true parameter is \((\gamma,\eta)=(0,\eta_0)\). Assume the model is identifiable and differentiable, the score has mean zero and finite nonsingular expected information, the usual likelihood regularity conditions hold, and the restricted estimator is consistent and interior. For independent observations, let \(I_n\) denote total expected information, with \(I_n/n\) converging to a positive-definite matrix \(I_0\). These conditions support the score central limit theorem and the local likelihood expansion.

At the true null parameter, the joint score divided by \(\sqrt n\) converges to a centered normal vector with covariance \(I_0\). A Taylor expansion of the nuisance score around \(\eta_0\), using \(U_\eta(0,\tilde\eta)=0\), gives \(\sqrt n(\tilde\eta-\eta_0)=I_{\eta\eta,0}^{-1}U_\eta(0,\eta_0)/\sqrt n+o_p(1)\). Expanding the tested score at the restricted estimate then yields \[ \frac{U_\gamma(0,\tilde\eta)}{\sqrt n} = \frac{U_\gamma(0,\eta_0)}{\sqrt n} -I_{\gamma\eta,0}I_{\eta\eta,0}^{-1} \frac{U_\eta(0,\eta_0)}{\sqrt n} +o_p(1). \] The leading term is a linear transformation of the joint asymptotic normal score. Its covariance is \(I_{\gamma\gamma,0}-I_{\gamma\eta,0}I_{\eta\eta,0}^{-1}I_{\eta\gamma,0}=I_{\gamma\gamma\cdot\eta,0}\). The information evaluated at the consistent restricted estimate, divided by \(n\), converges to the same matrix. Standardizing this \(q\)-dimensional normal limit by its covariance therefore gives a sum of squares of \(q\) independent standard normal variables. It follows that \(S\xrightarrow{d}\chi^2_q\). \(\square\)

Interpretation and Common Pitfalls

A large score statistic means that, at the best fit allowed by the null, the likelihood has a substantial slope in directions that would violate the restriction, relative to the information available in those directions. It does not require fitting the unrestricted model. In a generalized linear model, the score and expected information are evaluated using the restricted fitted means and the model’s variance and link structure.

• Using the unrestricted score. The unrestricted maximum has score zero, so it cannot provide the score test. Evaluate the score at the restricted maximum.
• Ignoring nuisance coefficients. The tested diagonal block of information is not generally the efficient information. Use the Schur complement, or the equivalent full-information contrast formula.
• Using the wrong degrees of freedom. The asymptotic chi-square degrees of freedom equal \(\operatorname{rank}(R)=q\), not the number of all coefficients in the model.
• Calling the test exact. The chi-square reference is generally justified asymptotically under regularity conditions; small samples or boundary and nonexistence problems require additional care.

The test is especially useful when estimating the unrestricted model is difficult or unstable, because it needs only the restricted fit and its score and information. Its construction also clarifies the role of nuisance parameters: their estimation changes the effective information even when their score is zero at the null fit.