Score Tests for Coefficients and Contrasts

Why the Null Fit Is Enough for a Score Test

A score test asks whether the likelihood is increasing away from a model that satisfies the null hypothesis. Its key practical feature is that it uses the restricted null fit: there is no need to fit the alternative model. For a single coefficient this can be a straightforward calculation, but a linear contrast requires care. The information for the tested direction must account for the directions used to estimate nuisance parameters.

We consider a regular GLM with parameter vector $\beta\in\mathbb{R}^p$, log-likelihood $\ell(\beta)$, score $U(\beta)=\partial\ell(\beta)/\partial\beta$, and expected Fisher information $I(\beta)=E[-\partial^2\ell(\beta)/\partial\beta\partial\beta^\mathsf{T}]$. All quantities in a score statistic below are evaluated at the null-constrained maximum likelihood estimate. We assume the null value is interior, the relevant information matrices are finite and nonsingular, and the usual regularity conditions for likelihood asymptotics hold.

Definition: For a scalar tested parameter $\psi$ and nuisance parameter vector $\gamma$, write the expected information at the restricted estimate in blocks as $I_{\psi\psi}, I_{\psi\gamma}, I_{\gamma\psi}, I_{\gamma\gamma}$. The efficient information for $\psi$, after accounting for estimation of $\gamma$, is $$ I_{\psi\cdot\gamma} =I_{\psi\psi}-I_{\psi\gamma}I_{\gamma\gamma}^{-1}I_{\gamma\psi}. $$ If the null hypothesis is $\psi=\psi_0$, and $\tilde\theta=(\psi_0,\tilde\gamma)$ is the restricted maximum likelihood estimate, the scalar score statistic is $$ S=\frac{U_\psi(\tilde\theta)^2}{I_{\psi\cdot\gamma}(\tilde\theta)}. $$ Under the null and the stated regularity conditions, $S\xrightarrow{d}\chi^2_1$.

At the restricted maximum, the nuisance score is zero: $U_\gamma(\tilde\theta)=0$. The score in the tested coordinate need not be zero; it measures the likelihood's remaining first-order change if the null constraint is relaxed. The adjustment in the denominator is essential whenever the tested coordinate and nuisance coordinates are not information-orthogonal.

Testing One Regression Coefficient

Suppose the coefficient of interest is $\beta_1$, with all remaining coefficients $\gamma$ treated as nuisance parameters. Fit the model under $H_0:\beta_1=0$, obtaining $(0,\tilde\gamma)$. Partition the score and information according to $(\beta_1,\gamma)$. Since nuisance parameters are maximized under the null, $U_\gamma(0,\tilde\gamma)=0$, so the statistic simplifies to

S =\frac{U_{\beta_1}(0,\tilde\gamma)^2} {I_{\beta_1\beta_1}-I_{\beta_1\gamma}I_{\gamma\gamma}^{-1}I_{\gamma\beta_1}}.

The numerator is the squared score for the coefficient at the null fit. The denominator is not generally just the diagonal entry $I_{\beta_1\beta_1}$: it is the information remaining after the nuisance directions have been accounted for.

Worked Example: A Poisson Regression Slope

Let $Y_i$ be independent Poisson responses with $\log E[Y_i]=\beta_0+\beta_1x_i$. For four observations, take $x=(0,0,1,1)$ and $y=(0,1,2,3)$. Test $H_0:\beta_1=0$.

Under the null, the fitted mean is the pooled sample mean, $\tilde\mu=(0+1+2+3)/4=1.5$, and $\tilde\beta_0=\log(1.5)$. For this canonical Poisson model, the score is $X^\mathsf{T}(y-\mu)$ and the expected information is $X^\mathsf{T}WX$, where $W$ has fitted means on its diagonal. The intercept score is $6-4(1.5)=0$. The slope score is $U_{\beta_1}=(2+3)-2(1.5)=2$.

Using columns $1$ and $x$, the information matrix at the null fit is

I =\begin{pmatrix}6&3\\3&3\end{pmatrix}.

The intercept is nuisance, so the slope's efficient information is $I_{\beta_1\cdot\beta_0}=3-3^2/6=1.5$. Therefore $S=2^2/1.5=8/3$, which is compared with a $\chi^2_1$ distribution. The score is evaluated at the intercept-only fit; no slope estimate under the alternative is needed.

Linear Contrasts: Choose a Tested Coordinate

Now consider $H_0:c^\mathsf{T}\beta=\psi_0$, where $c\in\mathbb{R}^p$ is nonzero. This is one scalar restriction, even if several coefficients appear in it. To apply the scalar formula, express the contrast as a parameter coordinate. Choose a vector $d$ and a matrix $B$ such that $c^\mathsf{T}d=1$, the columns of $B$ form a basis for the vectors $v$ satisfying $c^\mathsf{T}v=0$, and the matrix $[d\ B]$ is invertible. Write $\beta=d\psi+B\gamma$. Then $c^\mathsf{T}\beta=\psi$, so $\psi$ is exactly the tested coordinate and $\gamma$ contains the nuisance coordinates.

By the chain rule, the transformed score and information are $U_\psi=d^\mathsf{T}U_\beta$, $U_\gamma=B^\mathsf{T}U_\beta$, and $J=[d\ B]^\mathsf{T}I[d\ B]$. At the restricted fit $U_\gamma=0$. Thus the contrast score test is $U_\psi^2/J_{\psi\cdot\gamma}$, where $J_{\psi\cdot\gamma}=J_{\psi\psi}-J_{\psi\gamma}J_{\gamma\gamma}^{-1}J_{\gamma\psi}$.

Theorem: For a regular model with nonsingular information, the efficient information for the scalar contrast coordinate $\psi=c^\mathsf{T}\beta$ is $$ J_{\psi\cdot\gamma}=\frac{1}{c^\mathsf{T}I^{-1}c}. $$ Consequently, at the restricted estimate $\tilde\beta$, the score statistic for $H_0:c^\mathsf{T}\beta=\psi_0$ is $$ S_c =U_\psi(\tilde\beta)^2\,c^\mathsf{T}I(\tilde\beta)^{-1}c. $$

To verify the identity, let $A=[d\ B]$ and $J=A^\mathsf{T}IA$. The $(1,1)$ entry of $J^{-1}$ is the reciprocal of the Schur complement $J_{\psi\cdot\gamma}$, by the block inverse formula. The first row of $A^{-1}$ is $c^\mathsf{T}$, because $c^\mathsf{T}d=1$ and $c^\mathsf{T}B=0$. Hence the $(1,1)$ entry of $J^{-1}=A^{-1}I^{-1}(A^{-1})^\mathsf{T}$ is $c^\mathsf{T}I^{-1}c$. Equating the two expressions proves the identity. $\square$

The score $U_\psi=d^\mathsf{T}U_\beta$ is the score for the chosen contrast coordinate. It is not generally $c^\mathsf{T}U_\beta$. Although different valid choices of $d$ can differ by a nuisance direction, their scores agree at the restricted fit because $B^\mathsf{T}U_\beta=0$.

Worked Example: A Contrast with Correlated Coefficients

Consider a Gaussian linear model with known variance $\sigma^2=1$, no intercept, and two design columns $x_1=(1,-1,1,-1)^\mathsf{T}$ and $x_2=(1,-1,0,0)^\mathsf{T}$. The response is $y=(0.5,-0.5,0,0)^\mathsf{T}$. Test $H_0:\beta_1+\beta_2=0$, so $c=(1,1)^\mathsf{T}$ and $\psi=\beta_1+\beta_2$.

Use $\beta_1=\psi/2+\gamma$, $\beta_2=\psi/2-\gamma$, so $d=(1/2,1/2)^\mathsf{T}$ and the nuisance direction is $e=(1,-1)^\mathsf{T}$. Under the null, the fitted mean is $\gamma(x_1-x_2)$. Since $x_1-x_2=(0,0,1,-1)^\mathsf{T}$ and its inner product with $y$ is zero, the restricted estimate is $\tilde\gamma=0$; the null residual is $y$.

The score vector is $U_\beta=X^\mathsf{T}y=(1,1)^\mathsf{T}$: indeed $x_1^\mathsf{T}y=0.5+0.5=1$ and $x_2^\mathsf{T}y=0.5+0.5=1$. The information is $I=X^\mathsf{T}X=\begin{pmatrix}4&2\\2&2\end{pmatrix}$. The contrast-coordinate score is $U_\psi=d^\mathsf{T}U_\beta=(1/2,1/2)(1,1)^\mathsf{T}=1$, not $c^\mathsf{T}U_\beta=2$.

The transformed information entries are $J_{\psi\psi}=d^\mathsf{T}Id=2.5$, $J_{\psi\gamma}=d^\mathsf{T}Ie=1$, and $J_{\gamma\gamma}=e^\mathsf{T}Ie=2$. Therefore $J_{\psi\cdot\gamma}=2.5-1^2/2=2$, and $S_c=1^2/2=0.5$. Equivalently, $I^{-1}=\begin{pmatrix}0.5&-0.5\\-0.5&1\end{pmatrix}$, so $c^\mathsf{T}I^{-1}c=0.5$, giving the same statistic.

A Logistic Regression Contrast

Worked Example: Testing a Difference Between Two Group Log-Odds

Suppose independent binary outcomes are modeled by logistic regression with an intercept and a binary group indicator $x$. In group $x=0$, the outcomes are $1,0$; in group $x=1$, they are $1,1$. Test $H_0:\beta_1=0$, where $\beta_1$ is the group log-odds difference.

Under the null, the pooled success proportion is $3/4$, so $\tilde\beta_0=\log(3)$ and each fitted probability is $\tilde p=0.75$. For logistic regression the score is $X^\mathsf{T}(y-p)$, and the expected information is $X^\mathsf{T}WX$ with $w_i=p_i(1-p_i)$. The intercept score is $3-4(0.75)=0$. The slope score is the group-1 residual sum $2-2(0.75)=0.5$.

Every weight is $0.75(0.25)=3/16$. With two observations in each group, the information matrix, in intercept-slope order, is $I=\begin{pmatrix}3/4&3/8\\3/8&3/8\end{pmatrix}$. The efficient information for the slope is $3/8-(3/8)^2/(3/4)=3/16$. Thus $S=0.5^2/(3/16)=4/3$, compared with $\chi^2_1$. The adjustment accounts for estimating the intercept from the same observations.

How to Calculate and Interpret the Test

Fit the null model.
Estimate all nuisance parameters subject to the tested coefficient or contrast taking its null value.

Evaluate score and information there.
Compute the score in the tested coordinate and the expected information at the same restricted estimate.

Adjust for nuisance directions.
Use the information Schur complement, or for a contrast use $1/(c^\mathsf{T}I^{-1}c)$.

Form the statistic.
Divide the squared scalar score by its efficient information, then use the asymptotic $\chi^2_1$ reference distribution.

The result is an asymptotic test: under the null, with an identifiable interior parameter and regular likelihood behavior, the scalar score statistic converges in distribution to $\chi^2_1$. A large value indicates that the likelihood has a substantial slope away from the constrained fit in the tested direction. As with other likelihood-based large-sample tests, the approximation can be less reliable with very small samples, sparse data, or estimates near a boundary.

Common Pitfall: Using the Wrong Score or Information

A frequent error is to take the coefficient score at the null fit and divide its square by the corresponding diagonal information entry, without checking whether nuisance parameters are correlated with that coefficient. The correction is to use the efficient information $I_{\psi\cdot\gamma}$. For a contrast, another error is to use $c^\mathsf{T}U_\beta$ as its score merely because the contrast is $c^\mathsf{T}\beta$. The score is the derivative with respect to the chosen contrast coordinate, $U_\psi=d^\mathsf{T}U_\beta$; the nuisance-score equations make this value invariant to the allowed choice of $d$. In the worked Gaussian example, $c^\mathsf{T}U_\beta=2$, while the contrast-coordinate score is $1$. Substituting the former into the contrast statistic would give the wrong result.

Key takeaway: A score test for one coefficient or one linear contrast uses the score at the restricted null fit and divides its square by information adjusted for nuisance estimation. For a contrast, define the tested coordinate explicitly before identifying its score.

Check Your Understanding

Use the restricted fit for each calculation, and identify the tested coordinate before forming a contrast score.

For a single coefficient with score $U_{\beta_1}=3$, $I_{\beta_1\beta_1}=10$, and nuisance adjustment $I_{\beta_1\gamma}I_{\gamma\gamma}^{-1}I_{\gamma\beta_1}=1$, calculate the score statistic.
Why is the nuisance score zero at a regular restricted maximum likelihood estimate?
For a contrast $c^\mathsf{T}\beta$, what is the role of a vector $d$ satisfying $c^\mathsf{T}d=1$?
In the Gaussian contrast example, explain why using $c^\mathsf{T}U_\beta$ instead of $U_\psi$ would give an incorrect test statistic.
State the asymptotic reference distribution for a regular score test of one scalar restriction.