Score Test From the Restricted Model

Why Evaluate the Score at the Restricted Fit?

A score test asks whether the likelihood has a substantial slope away from the model imposed by the null hypothesis. It fits that restricted model, then measures the likelihood’s gradient in directions that would relax the restrictions. A large gradient indicates that the restricted fit sits on a steep part of the likelihood rather than near an unrestricted optimum.

For multivariate regression, the restrictions can link several coefficients and response contrasts. The matrix form keeps these restrictions organized, while the Gaussian likelihood supplies the score and the information needed to scale it. The resulting test uses the restricted maximum-likelihood estimate; it does not require fitting the unrestricted model.

Model and restriction: Let \(Y\) be an \(n\times q\) response matrix and \(X\) a fixed \(n\times p\) design matrix of full column rank. Assume the rows of \(Y\) are independent with distribution \(N_q((XB)_{i\cdot},\Sigma)\), where \(B\) is \(p\times q\) and \(\Sigma\) is positive definite. Test \(H_0:CBM=\Theta\), where \(C\) is \(r\times p\) of full row rank, \(M\) is \(q\times s\) of full column rank, and \(\Theta\) is \(r\times s\). This gives \(d=rs\) independent restrictions.

Write \(G=X'X\), \(E(B)=(Y-XB)'(Y-XB)\), and let \(\widetilde B,\widetilde\Sigma\) denote the joint restricted maximum-likelihood estimates under \(H_0\). Assume this restricted maximum exists at an interior point with \(\widetilde\Sigma\) positive definite. The Gaussian maximum-likelihood estimate of the covariance at a given coefficient matrix is \(E(B)/n\), so \(\widetilde\Sigma=E(\widetilde B)/n\).

Score and Information in the Gaussian Model

Ignoring constants that do not depend on the parameters, the log-likelihood is \(\ell(B,\Sigma)=-\frac n2\log|\Sigma|-\frac12\operatorname{tr}\{\Sigma^{-1}E(B)\}\). Differentiating with respect to \(B\) gives the \(p\times q\) coefficient score \(U(B,\Sigma)=X'(Y-XB)\Sigma^{-1}\). The expected Fisher information for \(\operatorname{vec}(B)\), with column-wise vectorization, is \(\Sigma^{-1}\otimes G\). The expected cross-information between \(B\) and \(\Sigma\) is zero in this Gaussian model.

U_0=X'(Y-X\widetilde B)\widetilde\Sigma^{-1},\qquad I_{B,0}=\widetilde\Sigma^{-1}\otimes G.

The null fit maximizes the likelihood over coefficient changes that preserve the restriction. Thus the coefficient score is zero in every such direction. It need not be zero in directions that violate the restriction; those remaining components are what the test uses. The covariance score is zero at the restricted maximum as well.

Definition: The expected-information score statistic for \(H_0:CBM=\Theta\) is \(T_S=\operatorname{vec}(U_0)'I_{B,0}^{-1}\operatorname{vec}(U_0)\), evaluated at the restricted maximum-likelihood estimates. Since \(I_{B,0}^{-1}=\widetilde\Sigma\otimes G^{-1}\), it can also be written \(T_S=\operatorname{tr}(U_0'G^{-1}U_0\widetilde\Sigma)\).

The restriction matrix in vectorized form is \(A=M'\otimes C\), because \(\operatorname{vec}(CBM)=A\operatorname{vec}(B)\). It has rank \(d=rs\). Consequently the allowed coefficient changes under the null form the null space of \(A\). At the restricted fit, stationarity says that \(\operatorname{vec}(U_0)\) is orthogonal to this null space, so it lies in the row space of \(A\). The statistic therefore measures the score only in the directions tested by the restrictions.

Deriving the Score Statistic

The following argument shows why the quadratic form above is the score test for the stated restrictions, rather than merely a convenient measure of score size. It also identifies the scale attached to the tested directions.

Identify the tested and untested directions.
Let \(b=\operatorname{vec}(B)\) and \(u_0=\operatorname{vec}(U_0)\). The null tangent directions are vectors \(v\) satisfying \(Av=0\). Stationarity of the restricted likelihood gives \(v'u_0=0\) for every such \(v\).

Use the orthogonal-complement relation.
For a full-row-rank matrix \(A\), the orthogonal complement of its null space is its row space. Hence \(u_0=A'\lambda\) for some \(d\)-vector \(\lambda\).

Scale the score by information.
The full coefficient information is \(I_{B,0}\). Substituting \(u_0=A'\lambda\) into the score quadratic gives \(u_0'I_{B,0}^{-1}u_0=\lambda'(A I_{B,0}^{-1}A')\lambda\), the score quadratic in the \(d\) tested directions.

This calculation also explains why nuisance coefficient directions must not be included as if they were tested. Their score components vanish at the restricted maximum. The covariance parameter is a nuisance parameter too, but its expected cross-information with the coefficient parameter is zero here, and its score vanishes at the restricted maximum.

Theorem: Suppose \(X\) has full column rank, \(C\) and \(M\) have the stated ranks, the true covariance is positive definite, and the restricted maximum-likelihood estimates are interior and regular. Under \(H_0\), for fixed \(p,q,r,s\) and a sequence of designs with \(G/n\) converging to a positive-definite matrix, \(T_S\xrightarrow{d}\chi^2_d\), where \(d=rs\).

To justify the reference distribution, reparameterize the coefficients into \(d\) coordinates for departures from the null and coordinates tangent to the null; write the covariance coordinates among the nuisance parameters. Let \(S_\gamma\) be the score for departures from the null and \(S_\zeta\) the nuisance score. Write the Fisher information in matching blocks. The efficient score for the tested coordinates is \(S_{\mathrm{eff}}=S_\gamma-I_{\gamma\zeta}I_{\zeta\zeta}^{-1}S_\zeta\), with efficient information \(I_{\mathrm{eff}}=I_{\gamma\gamma}-I_{\gamma\zeta}I_{\zeta\zeta}^{-1}I_{\zeta\gamma}\).

Under the null, the Gaussian likelihood is regular, the information grows proportionally to \(n\), and the score central limit theorem gives the joint asymptotic normal distribution of the scaled scores. A Taylor expansion of the nuisance score equation around the true parameter shows that fitting the nuisance parameters under the null subtracts the nuisance projection above. Thus \(I_{\mathrm{eff}}^{-1/2}S_{\mathrm{eff}}\xrightarrow{d}N_d(0,I_d)\). The restricted score quadratic is the squared length of this standardized efficient score, up to a term that converges to zero in probability. By the continuous mapping theorem, its limit is \(\chi^2_d\). The Gaussian coefficient–covariance cross-information is zero, and the stationarity argument above reduces this quadratic to \(T_S\). Therefore \(T_S\xrightarrow{d}\chi^2_d\). \(\square\)

Worked Calculations

Worked Example: Testing One Slope in a Single Response

Take a Gaussian regression with an intercept and one predictor, \(x=(-1,-1,-1,1,1,1)'\), and response \(y=2\mathbf 1+0.5x+e\), where \(e=(\sqrt6,-\sqrt6,0,0,0,0)'\). Test the slope restriction \(H_0:\beta_x=0\). Here \(x'\mathbf 1=0\), \(x'x=6\), \(\mathbf 1'e=0\), and \(x'e=0\).

Under the restricted fit, the intercept estimate is \(\bar y=2\). The restricted residual is \(0.5x+e\), so its residual sum of squares is \(0.25(x'x)+e'e=0.25(6)+12=13.5\). Therefore \(\widetilde\sigma^2=13.5/6=2.25\).

The slope score is \(x'(y-2\mathbf 1)/\widetilde\sigma^2\). Its numerator is \(x'(0.5x+e)=0.5(6)+0=3\), so \(U_0=3/2.25=4/3\). The expected information for the slope is \(x'x/\widetilde\sigma^2=6/2.25=8/3\). Thus \(T_S=(4/3)^2/(8/3)=2/3\). The test has one restriction, so its large-sample reference distribution is \(\chi^2_1\); the value \(2/3\) is not evidence of a large score against the restricted model.

Worked Example: Testing a Predictor Across Two Responses

Let \(n=6\), \(q=2\), and use an intercept and \(x=(-1,-1,-1,1,1,1)'\), so \(G=\operatorname{diag}(6,6)\). Choose vectors \(z=\sqrt3(1,-1,0,0,0,0)'\) and \(w=\sqrt3(0,0,0,1,-1,0)'\); each is orthogonal to both the intercept and \(x\), has squared length 6, and \(z'w=0\). Let the response columns be \(Y_1=0.5x+z\) and \(Y_2=x+2w\). Test that the predictor’s coefficient is zero for both responses.

The restricted intercept estimates are both zero. The restricted residual sum-of-products matrix is \(E_0=Y'Y=\begin{pmatrix}7.5&3\\3&30\end{pmatrix}\), so \(\widetilde\Sigma=E_0/6=\begin{pmatrix}1.25&0.5\\0.5&5\end{pmatrix}\). Its determinant is \(1.25(5)-0.5^2=6\), and \(\widetilde\Sigma^{-1}=\frac16\begin{pmatrix}5&-0.5\\-0.5&1.25\end{pmatrix}\).

The predictor-row score is \(x'Y\widetilde\Sigma^{-1}\). Since \(x'Y=(3,6)\), this equals \((2,1)\): multiplying gives \(((15-3)/6,(-1.5+7.5)/6)=(2,1)\). The inverse information for that coefficient row is \(\widetilde\Sigma/6\). Consequently \(T_S=(2,1)(\widetilde\Sigma/6)(2,1)'=2\), because \((2,1)\widetilde\Sigma(2,1)'=12\). The restriction has \(d=2\) degrees of freedom, one for each response coefficient.

Worked Example: A Coupled Predictor and Response Contrast

Suppose \(p=q=2\), \(G=\operatorname{diag}(4,9)\), and the restricted fit has \(\widetilde\Sigma=I_2\) and coefficient score \(U_0=\begin{pmatrix}2&2\\-2&-2\end{pmatrix}\). Test the single restriction \(CBM=0\), where \(C=(1,-1)\) and \(M=(1,1)'\). The restriction compares the two predictor coefficients after taking the sum of the two response components.

Here \(A=M'\otimes C=(1,-1,1,-1)\), and \(\operatorname{vec}(U_0)=(2,-2,2,-2)'=2A'\). Thus the score has only the tested-direction component, as required at a restricted maximum. The inverse information is \(I_2\otimes G^{-1}\), with \(G^{-1}=\operatorname{diag}(1/4,1/9)\). Therefore \(T_S=2^2(1/4+1/9+1/4+1/9)=4(13/18)=26/9\). There is one independent restriction, so the asymptotic reference distribution is \(\chi^2_1\).

Interpretation and a Common Pitfall

The score test is useful when fitting the unrestricted model is inconvenient or when the scientific question is naturally posed at a restricted fit. Its statistic combines two things: how large the likelihood gradient is in tested directions, and how much variability the information predicts in those directions. A raw score alone is not comparable across coefficient scales or predictor designs.

Pitfall: treating a nonzero restricted score as a test statistic by itself. A score of a given numerical size may be large under one design and ordinary under another. The correction is to scale the score by the inverse expected information, as in \(T_S\).
Pitfall: using all coefficient-score entries without checking the restriction. At the restricted maximum, nuisance-direction scores are zero by stationarity; only directions that violate the restrictions contribute. The row-space argument establishes this precisely.
Pitfall: reporting a chi-square law as exact in a small sample. The chi-square reference here is asymptotic under the stated regularity conditions. It is not a claim that the statistic has that exact finite-sample distribution.

Key takeaway: Fit the Gaussian model under the restrictions, evaluate its coefficient score and expected information there, and form their information-scaled quadratic. Under regularity and the null, this score statistic converges to a chi-square distribution with one degree of freedom per independent coefficient restriction.

Check Your Understanding

Answer using the model, restriction, and score-test construction given above.

For \(CBM=\Theta\), why is the number of independent restrictions \(rs\) under the stated rank conditions?
What stationarity condition makes the restricted coefficient score lie in the row space of \(M'\otimes C\)?
In the two-response example, verify the determinant of \(\widetilde\Sigma\) and the restricted predictor-row score.
Why is the inverse expected information needed to scale the score?
State two conditions that support the asymptotic chi-square reference distribution, and explain why it should not automatically be treated as exact in small samples.