The Score Test for Vector Restrictions

Why a Vector Score Test Is Joint

A single score component measures how the log-likelihood initially changes in one parameter direction under the null. When several restrictions are tested together, the relevant question is whether the likelihood has a large initial slope in any combination of those directions. The components may be correlated, so their joint scale cannot generally be found by treating each one as a separate test and adding the results.

The vector score test evaluates the score at the fit constrained by the null hypothesis, then standardizes the score using the information matrix. When nuisance parameters are estimated under the null, the information used for standardization must also account for them.

Definition and Construction

Definition: Let $X_1,\ldots,X_n$ have a regular likelihood $L(\theta)$, with log-likelihood $\ell(\theta)=\log L(\theta)$. Partition the parameter as $\theta=(\psi,\lambda)$, where $\psi$ is an $m$-dimensional parameter of interest and $\lambda$ is a nuisance parameter. To test $H_0:\psi=\psi_0$, let $\tilde{\lambda}$ be the maximum-likelihood estimate of $\lambda$ under $H_0$. Evaluate the score at $\tilde{\theta}_0=(\psi_0,\tilde{\lambda})$. If there is no nuisance parameter, or it is known, the score statistic is $$ S = U_\psi(\psi_0)^{\mathsf T} I_{\psi\psi}(\psi_0)^{-1} U_\psi(\psi_0), $$ where $U_\psi=\partial\ell/\partial\psi$ is the score vector and $I_{\psi\psi}$ is its Fisher information matrix. With an estimated nuisance parameter, use $$ S = U_\psi(\tilde{\theta}_0)^{\mathsf T} I_{\psi\psi\cdot\lambda}(\tilde{\theta}_0)^{-1} U_\psi(\tilde{\theta}_0), \qquad I_{\psi\psi\cdot\lambda} = I_{\psi\psi}-I_{\psi\lambda}I_{\lambda\lambda}^{-1}I_{\lambda\psi}. $$ Here $I_{\psi\psi\cdot\lambda}$ is the information for $\psi$ adjusted for estimating $\lambda$.

The information matrix is the covariance matrix of the full score under the model, under the usual regularity conditions. Its off-diagonal entries record dependence among score components. In the nuisance-parameter case, the displayed Schur complement accounts for the part of the score variation that is entangled with nuisance-parameter estimation.

The information in the definition is expected Fisher information, evaluated at the restricted fit. An observed-information version replaces it with the negative Hessian of the log-likelihood at that fit and uses the corresponding Schur complement. Under standard regularity conditions, the two versions give the same first-order large-sample calibration.

Why the Nuisance Adjustment Has This Form

Partition the full score and information at the restricted fit as $U=(U_\psi^{\mathsf T},U_\lambda^{\mathsf T})^{\mathsf T}$ and $I=\begin{pmatrix}I_{\psi\psi}&I_{\psi\lambda}\\I_{\lambda\psi}&I_{\lambda\lambda}\end{pmatrix}$. The score for $\lambda$ is zero at an interior restricted maximum, so $U_\lambda=0$. The score component relevant after allowing nuisance parameters to adjust can be written $U_{\psi\cdot\lambda}=U_\psi-I_{\psi\lambda}I_{\lambda\lambda}^{-1}U_\lambda$; at that restricted fit it equals $U_\psi$. Its covariance is the Schur complement.

Result: If $I_{\lambda\lambda}$ is nonsingular, the covariance matrix of $U_{\psi\cdot\lambda}$ is $I_{\psi\psi\cdot\lambda}$. In particular, using $I_{\psi\psi}$ alone does not generally give the correct standardization when nuisance parameters are present.

To verify the covariance claim, use $\operatorname{Cov}(U)=I$ and set $A=I_{\psi\lambda}I_{\lambda\lambda}^{-1}$. Then $U_{\psi\cdot\lambda}=U_\psi-AU_\lambda$. Expanding its covariance gives $$ \begin{aligned} \operatorname{Var}(U_\psi-AU_\lambda) &= I_{\psi\psi}-I_{\psi\lambda}A^{\mathsf T} -AI_{\lambda\psi}+AI_{\lambda\lambda}A^{\mathsf T}\\ &= I_{\psi\psi}-I_{\psi\lambda}I_{\lambda\lambda}^{-1}I_{\lambda\psi} = I_{\psi\psi\cdot\lambda}. \end{aligned} $$ The second equality follows because the information matrix is symmetric, so $A^{\mathsf T}=I_{\lambda\lambda}^{-1}I_{\lambda\psi}$. This adjustment is the multivariate counterpart of accounting for estimation of a nuisance parameter in a scalar score test. $\square$

For a fixed number $m$ of independent restrictions, under the null and standard regularity conditions—including an interior null value, differentiability, identifiable parameters, a consistent restricted maximum-likelihood estimate, and nonsingular information—the statistic is asymptotically calibrated by a chi-square distribution with $m$ degrees of freedom. The test is upper-tailed: large values provide evidence against the null. This calibration is a large-sample approximation, not a guarantee of accuracy in a small sample.

Worked Examples

Worked Example: Two Correlated Normal-Mean Restrictions

Let $X_1,\ldots,X_{27}$ be independent bivariate normal observations with known covariance matrix $\Sigma=\begin{pmatrix}4&3\\3&9\end{pmatrix}$ and unknown mean vector $\mu=(\mu_1,\mu_2)^{\mathsf T}$. Test $H_0:\mu=(0,0)^{\mathsf T}$. Suppose the observed sample mean is $\bar X=(1,2)^{\mathsf T}$.

The score for $\mu$ at the null is $U=27\Sigma^{-1}\bar X$. Since $\det(\Sigma)=4(9)-3(3)=27$, $\Sigma^{-1}=\frac{1}{27}\begin{pmatrix}9&-3\\-3&4\end{pmatrix}$, and therefore $$ U=27\left[\frac{1}{27} \begin{pmatrix}9&-3\\-3&4\end{pmatrix} \begin{pmatrix}1\\2\end{pmatrix}\right] =\begin{pmatrix}3\\5\end{pmatrix}. $$ The information is $I=27\Sigma^{-1}$, whose inverse is $I^{-1}=\Sigma/27$. Hence $$ S=U^{\mathsf T}I^{-1}U =\frac{1}{27} \begin{pmatrix}3&5\end{pmatrix} \begin{pmatrix}4&3\\3&9\end{pmatrix} \begin{pmatrix}3\\5\end{pmatrix} =\frac{1}{27}(3\cdot27+5\cdot54)=13. $$ The two score components are not independent: their covariance is the off-diagonal element $-3$ of $I$. The joint statistic uses that information. Under the stated large-sample calibration, $S$ is compared with a chi-square distribution with two degrees of freedom.

Worked Example: Two Multinomial Probability Restrictions

In $n=100$ independent trials, let the three category probabilities be $p_1,p_2,p_3$, with $p_3=1-p_1-p_2$. Test $H_0:(p_1,p_2)=(0.2,0.3)$, so $p_3=0.5$ under the null. The observed counts are $(18,34,48)$.

Ignoring the multinomial coefficient, the log-likelihood is $\ell=p_1\log p_1+p_2\log p_2+p_3\log p_3$ with the observed counts as coefficients. Differentiating with respect to the two free probabilities gives the score components $U_1=x_1/p_1-x_3/p_3$ and $U_2=x_2/p_2-x_3/p_3$. At the null, $$ U_1=\frac{18}{0.2}-\frac{48}{0.5}=-6,\qquad U_2=\frac{34}{0.3}-\frac{48}{0.5}=\frac{52}{3}. $$ The expected information is $$ I=100\begin{pmatrix} 1/p_1+1/p_3&1/p_3\\ 1/p_3&1/p_2+1/p_3 \end{pmatrix} =\begin{pmatrix}700&200\\200&1600/3\end{pmatrix}. $$ Its inverse at the null is $I^{-1}=\frac{1}{100}\begin{pmatrix}0.16&-0.06\\-0.06&0.21\end{pmatrix}$. Thus $$ \begin{aligned} S &=\frac{1}{100}\left[ 0.16(36)+2(-0.06)\left(-6\right)\left(\frac{52}{3}\right) +0.21\left(\frac{52}{3}\right)^2 \right]\\ &=\frac{1}{100}(5.76+12.48+63.093\overline{3}) =\frac{61}{75}\approx 0.813. \end{aligned} $$ The off-diagonal information entries are nonzero, so the cross-product term contributes to the result. The statistic is a joint test of both restrictions; neither score component should be standardized and tested in isolation as a substitute.

Worked Example: Testing Two Regression Slopes with a Nuisance Intercept

Consider the normal linear model $Y_i=\alpha+\beta_1x_{i1}+\beta_2x_{i2}+\varepsilon_i$, with independent errors $\varepsilon_i\sim N(0,1)$. The intercept $\alpha$ is a nuisance parameter. For four observations, take $x_1=(-1,-1,1,1)^{\mathsf T}$, $x_2=(-2,0,0,2)^{\mathsf T}$, and observed responses $y=(0,1,2,5)^{\mathsf T}$. Test $H_0:(\beta_1,\beta_2)=(0,0)$.

Under the null, the restricted estimate of the intercept is $\tilde\alpha=\bar y=2$. Both predictor vectors sum to zero, so the slope score $X^{\mathsf T}(y-\tilde\alpha\mathbf{1})$ equals $X^{\mathsf T}y$. Its components are $U_1=0-1+2+5=6$ and $U_2=0+0+0+10=10$. The slope information is $X^{\mathsf T}X$: $$ I_{\beta\beta} =\begin{pmatrix}x_1^{\mathsf T}x_1&x_1^{\mathsf T}x_2\\ x_2^{\mathsf T}x_1&x_2^{\mathsf T}x_2\end{pmatrix} =\begin{pmatrix}4&4\\4&8\end{pmatrix}. $$ Because the predictors are centered, their information cross-terms with the intercept are zero; therefore nuisance adjustment leaves this slope information unchanged. Its inverse is $\frac{1}{16}\begin{pmatrix}8&-4\\-4&4\end{pmatrix}$, giving $$ S=\begin{pmatrix}6&10\end{pmatrix} \frac{1}{16}\begin{pmatrix}8&-4\\-4&4\end{pmatrix} \begin{pmatrix}6\\10\end{pmatrix} =18-30+25=13. $$ If the off-diagonal information were wrongly ignored, the result would be $6^2/4+10^2/8=21.5$, not $13$. The example shows both why the score components must be combined and how nuisance adjustment is checked.

Common Mistakes and Interpretation

Adding separate one-dimensional statistics. This ignores covariance between score components. Use the full inverse information matrix, including its off-diagonal entries.
Evaluating the score at the unrestricted estimate. The score is zero at an interior unrestricted maximum. A score test evaluates it at the null-constrained fit.
Using $I_{\psi\psi}$ when nuisance parameters are estimated. Use the Schur complement $I_{\psi\psi\cdot\lambda}$, unless the information cross-block is zero or the nuisance parameter is known.
Treating the chi-square reference as exact. It is an asymptotic calibration. With small samples, its tail probabilities may be inaccurate.

The score test is especially useful when fitting the unrestricted model is inconvenient: it requires the restricted fit and the score and information there. For a vector null, its main conceptual safeguard is also its main computational rule: retain the entire score vector and standardize it with the joint information matrix.

Check Your Understanding

Work through these questions using the definitions and calculations above.

For a two-component score with nonsingular information matrix $I$, why is the joint statistic not generally the sum of the two squared scores divided by their separate information entries?
In a model partitioned into tested parameter $\psi$ and nuisance parameter $\lambda$, write the information matrix used to standardize the score for $\psi$.
Why is the nuisance score $U_\lambda$ zero at an interior restricted maximum-likelihood estimate?
In the multinomial example, identify the information-matrix entry that records dependence between the two score components.
What conditions make the chi-square calibration an approximation rather than an exact finite-sample result?