Joint Score Tests Under Restricted GLM Fits

Why a Joint Score Test Uses a Restricted Fit

A joint score test asks whether several coefficients can be set to specified null values, using the model fitted under that null. This can be useful when the unrestricted model is difficult to fit or when the question is specifically whether a block of effects contributes beyond the other terms. The essential calculation is not just a score vector: it also needs the information for those effects after accounting for the nuisance coefficients estimated in the restricted model.

Write the parameter as $\theta=(\gamma^\top,\delta^\top)^\top$, where $\gamma$ contains nuisance coefficients and $\delta$ is a block of $q$ coefficients to test. For clarity, take the null hypothesis to be $H_0:\delta=0$. Under the null, maximize the likelihood over $\gamma$ only, obtaining the restricted maximum-likelihood estimate $\hat{\gamma}_0$. The score for $\delta$ is then evaluated at $(\hat{\gamma}_0^\top,0^\top)^\top$.

Definition: Let $U(\theta)=\partial\ell(\theta)/\partial\theta$ be the score and let $I(\theta)$ be the expected Fisher information. Partition the information into blocks corresponding to $\gamma$ and $\delta$. The restricted-fit score is $U_\delta(\hat{\gamma}_0,0)$. Its nuisance-adjusted information is the Schur complement $$ I_{\delta\delta\cdot\gamma} = I_{\delta\delta} - I_{\delta\gamma}I_{\gamma\gamma}^{-1}I_{\gamma\delta}, $$ with all blocks evaluated at $(\hat{\gamma}_0,0)$. The joint score statistic is $$ S = U_\delta(\hat{\gamma}_0,0)^\top I_{\delta\delta\cdot\gamma}^{-1} U_\delta(\hat{\gamma}_0,0). $$

This formula assumes $I_{\gamma\gamma}$ and $I_{\delta\delta\cdot\gamma}$ are nonsingular. The subtraction matters: the information in the tested block is adjusted for the fact that the nuisance coefficients have been estimated, rather than known.

Computing the Statistic from the Restricted Fit

For a joint test, the statistic combines all $q$ components of the restricted-fit score. Under regularity conditions stated below, a large value is evidence against the null. The reference distribution has $q$ degrees of freedom: the number of independent coefficient restrictions, not the number of nuisance coefficients.

Fit the null model.
Set the tested coefficients to their null values and maximize the likelihood over the remaining coefficients.

Evaluate the tested score and information.
At the restricted estimate, calculate $U_\delta$ and the four information blocks.

Adjust for nuisance estimation.
Form $I_{\delta\delta\cdot\gamma}=I_{\delta\delta}-I_{\delta\gamma}I_{\gamma\gamma}^{-1}I_{\gamma\delta}$.

Form the quadratic statistic.
Compute $S=U_\delta^\top I_{\delta\delta\cdot\gamma}^{-1}U_\delta$ and compare it with a chi-square distribution with $q$ degrees of freedom.

Worked Example: Joint Poisson Test with Two Slopes

Suppose four independent Poisson responses have means $\mu_i=\exp(\alpha+\beta_1x_{i1}+\beta_2x_{i2})$, with predictor rows $(0,0),(1,0),(0,1),(1,1)$ and observed counts $y=(0,1,2,5)$. Test $H_0:\beta_1=\beta_2=0$. Under the null, the common fitted mean is $\hat{\mu}=\bar{y}=2$, so $\hat{\alpha}_0=\log 2$.

For a Poisson log-link model, the score for a slope is the sum of its predictor multiplied by $y_i-\mu_i$. For $x_1$, the rows with $x_1=1$ are rows 2 and 4, so $(1-2)+(5-2)=2$. For $x_2$, the rows with $x_2=1$ are rows 3 and 4, so $(2-2)+(5-2)=3$. Thus $U_\delta=(2,3)^\top$.

The expected information for Poisson regression is $X^\top W X$, with weights $\mu_i=2$. In the order $(\alpha,\beta_1,\beta_2)$, the information blocks at the restricted fit are

I_{\gamma\gamma}=8,\qquad I_{\delta\gamma}= \begin{pmatrix}4\\4\end{pmatrix},\qquad I_{\delta\delta}= \begin{pmatrix}4&2\\2&4\end{pmatrix}.

Therefore

I_{\delta\delta\cdot\gamma} = \begin{pmatrix}4&2\\2&4\end{pmatrix} - \frac{1}{8} \begin{pmatrix}4\\4\end{pmatrix} \begin{pmatrix}4&4\end{pmatrix} = \begin{pmatrix}2&0\\0&2\end{pmatrix}.

Its inverse is $\frac12$ times the identity matrix, giving $S=(2^2+3^2)/2=13/2=6.5$. The asymptotic reference distribution is $\chi^2_2$. For comparison, using the unadjusted slope-information block would give \[ (2,3) \left\{\frac{1}{12} \begin{pmatrix}4&-2\\-2&4\end{pmatrix}\right\} \begin{pmatrix}2\\3\end{pmatrix} = \frac{28}{12} = \frac{7}{3}, \] which is not the joint score statistic. These small data are used to show the calculation, not to guarantee a reliable chi-square approximation.

Why the Information Must Be Adjusted

The adjustment has a direct score-based interpretation. Under the null, the nuisance score is zero at the restricted maximum-likelihood estimate, but the nuisance coefficients are still estimated from the data. Changes in $\gamma$ can therefore be associated with changes in the tested score. The Schur complement removes the part of the tested block's information attributable to that association.

The same adjustment appears in the variance of the score after accounting for nuisance estimation. In the theorem below, $I_n$ denotes the total expected information for a sample of size $n$. Assume a fixed-dimensional, regular, identifiable model; an interior true parameter under $H_0$; differentiable log-likelihood with a valid local quadratic expansion; and nonsingular limiting information per observation. Assume also the score central limit theorem and the usual consistency and asymptotic linearity of the restricted maximum-likelihood estimator. Independent observations with suitable moment and Lindeberg conditions are a standard setting in which these assumptions can hold. The dimension $q$ is fixed as $n$ increases.

Theorem: Under these conditions, if the true parameter is $(\gamma_0^\top,0^\top)^\top$, then the restricted-fit joint score statistic satisfies $S\xrightarrow{d}\chi^2_q$.

Proof. Evaluate the total score and information at the true null parameter, and denote the information blocks there by $I_{\gamma\gamma,n}$, $I_{\delta\gamma,n}$, and so on. The restricted estimator has the first-order expansion

\hat{\gamma}_0-\gamma_0 = I_{\gamma\gamma,n}^{-1}U_\gamma(\gamma_0,0)+o_p(n^{-1/2}).

A Taylor expansion of the tested score in the nuisance parameter, using the score derivative equal to minus the information to first order, gives

U_\delta(\hat{\gamma}_0,0) = U_\delta(\gamma_0,0) - I_{\delta\gamma,n}(\hat{\gamma}_0-\gamma_0) + o_p(\sqrt n) = U_\delta-I_{\delta\gamma,n}I_{\gamma\gamma,n}^{-1}U_\gamma+o_p(\sqrt n).

The leading term is a linear transformation of the joint score. Under the null, the joint score has asymptotic covariance $I_n$. Direct multiplication of its block covariance matrix shows that the covariance of this transformed score is

I_{\delta\delta,n} - I_{\delta\gamma,n}I_{\gamma\gamma,n}^{-1}I_{\gamma\delta,n} = I_{\delta\delta\cdot\gamma,n}.

Consequently the adjusted score, divided by $\sqrt n$, converges in distribution to a mean-zero normal vector with covariance $J_{\delta\delta\cdot\gamma}$, where $I_{\delta\delta\cdot\gamma,n}/n\to J_{\delta\delta\cdot\gamma}$, assumed positive definite. The information evaluated at the restricted estimate has this same limit after division by $n$, by consistency and continuity. Applying Slutsky's theorem to the quadratic form gives the squared length of a $q$-dimensional standard normal vector, which has a $\chi^2_q$ distribution. $\square$

Worked Example: Logistic Test with an Intercept as Nuisance

Consider four independent Bernoulli responses with logistic model $\operatorname{logit}(p_i)=\alpha+\beta_1x_{i1}+\beta_2x_{i2}$, using predictor rows $(0,0),(1,0),(0,1),(1,1)$ and outcomes $y=(0,0,0,1)$. Under $H_0:\beta_1=\beta_2=0$, the restricted estimate is $\hat{p}=1/4$, so $\hat{\alpha}_0=\log(1/3)$.

The tested scores are the sums of $x_{ij}(y_i-\hat{p})$. Each predictor is 1 in two rows, one of which has outcome 1, so each score is $1-2(1/4)=1/2$. Hence $U_\delta=(1/2,1/2)^\top$. The logistic information weights are $\hat{p}(1-\hat{p})=3/16$. Thus $I_{\gamma\gamma}=4(3/16)=3/4$, $I_{\delta\gamma}=(3/8,3/8)^\top$, and

I_{\delta\delta} = \begin{pmatrix}3/8&3/16\\3/16&3/8\end{pmatrix}, \qquad I_{\delta\delta\cdot\gamma} = \begin{pmatrix}3/16&0\\0&3/16\end{pmatrix}.

It follows that $S=2(1/2)^2/(3/16)=8/3$, with an asymptotic $\chi^2_2$ reference distribution. As with the Poisson calculation, the example verifies the construction; the small sample itself may not support a good asymptotic approximation.

Worked Example: A Gaussian Model with Known Variance

Let $Y_i$ be independent normal responses with variance 1 and mean $\alpha+\beta_1x_{i1}+\beta_2x_{i2}$. Use the same four predictor rows as above, but let $y=(0,1,2,5)$. Under $H_0:\beta_1=\beta_2=0$, the restricted estimate is $\hat{\alpha}_0=\bar{y}=2$, and the residuals are $(-2,-1,0,3)$.

The tested score is $X_\delta^\top(y-\hat{\alpha}_0\mathbf{1})$. Summing residuals in rows where $x_1=1$ gives $-1+3=2$; summing where $x_2=1$ gives $0+3=3$. Thus $U_\delta=(2,3)^\top$. With known variance 1, the information is the design cross-product. The intercept information is 4, the intercept-slope block is $(2,2)$, and the slope block is $\begin{pmatrix}2&1\\1&2\end{pmatrix}$. Adjusting for the intercept gives

I_{\delta\delta\cdot\gamma} = \begin{pmatrix}2&1\\1&2\end{pmatrix} - \frac14 \begin{pmatrix}2\\2\end{pmatrix} \begin{pmatrix}2&2\end{pmatrix} = \begin{pmatrix}1&0\\0&1\end{pmatrix}.

Therefore $S=2^2+3^2=13$, with a $\chi^2_2$ asymptotic reference distribution. The score and adjusted information are both computed at the restricted fit; neither requires fitting the two-slope alternative.

Common Pitfall and Practical Interpretation

Wrong reasoning: “The score has two components, so I can use the $2\times2$ block $I_{\delta\delta}$ directly in the quadratic form.” This treats the nuisance coefficients as though they were known. When nuisance estimates are obtained from the same data, the relevant information is the Schur complement, not the raw block.

A second practical check is to keep the score components aligned with their predictor columns. In the Poisson example, the $x_1=1$ rows have outcomes 1 and 5, giving score 2; the $x_2=1$ rows have outcomes 2 and 5, giving score 3. Interchanging these scores happens not to change that example's statistic because its adjusted information is proportional to the identity, but that coincidence need not hold in a different design.

The joint test answers whether the tested coefficient block is compatible with its null values, conditional on the nuisance part being fitted under the null. It is an asymptotic test: a chi-square reference can be inaccurate in small samples, with sparse outcomes, or near nonregular parameter configurations. Its computational advantage is specific and useful: it requires the restricted fit and the score and information there, rather than maximizing the unrestricted likelihood.

Check Your Understanding

Use the definitions and calculations above to check that you can construct and interpret the joint test.

For a null hypothesis setting three coefficients to zero, how many degrees of freedom does the regular joint score test use?
Why is the restricted-fit score evaluated at the nuisance estimate rather than at an arbitrary nuisance value?
State the nuisance-adjusted information for a tested block $\delta$ and nuisance block $\gamma$.
In the Poisson example, verify the two slope scores from the predictor rows and outcomes.
What assumption is made when one uses $I_{\delta\delta}$ directly instead of the Schur complement?

Joint Score Tests Under Restricted GLM Fits

What You'll Learn