Why Test From the Restricted Fit?
A likelihood test need not begin by fitting the model without restrictions. To test whether one scalar parameter equals a specified value, the score test asks whether the likelihood is increasing away from that value, using its slope and curvature at the restricted fit. This can be convenient when imposing the null makes estimation simple, even if the unrestricted model is difficult to fit.
When nuisance parameters are present, the slope for the parameter of interest must be assessed after accounting for the nuisance parameters. The information used in the denominator is therefore not generally just the information for the scalar parameter by itself.
\(I(\theta)=\begin{pmatrix} I_{\psi\psi} & I_{\psi\lambda}\\ I_{\lambda\psi} & I_{\lambda\lambda}\end{pmatrix}.\)
The efficient score for \(\psi\), with nuisance score removed, is \(U_{\mathrm{eff}}=U_\psi-I_{\psi\lambda}I_{\lambda\lambda}^{-1}U_\lambda\). The corresponding efficient information is \(I_{\mathrm{eff}}=I_{\psi\psi}-I_{\psi\lambda}I_{\lambda\lambda}^{-1}I_{\lambda\psi}\), with all quantities evaluated at \((\psi_0,\tilde{\lambda})\). The scalar score statistic is \(S=U_{\mathrm{eff}}^2/I_{\mathrm{eff}}\).
If the restricted maximum is an interior point in the nuisance parameters, its nuisance score satisfies \(U_\lambda(\psi_0,\tilde{\lambda})=0\). In that common case, the numerator simplifies to \(U_\psi^2\), but the denominator remains the nuisance-adjusted information \(I_{\mathrm{eff}}\). The score is a derivative of the log-likelihood; it is not itself a test statistic until it has been standardized by the appropriate information.
Why the Information Must Account for Nuisance Parameters
The adjustment has a precise interpretation. The nuisance score may move in the same direction as the score for \(\psi\). The part of \(U_\psi\) predictable from \(U_\lambda\) does not provide independent evidence about \(\psi\), so it is removed. The information left over is the variance of that adjusted score.
To verify the variance statement, take information at a fixed parameter value and use \(\operatorname{Cov}(U)=I\), which holds for a regular model with mean-zero score. Put \(A=I_{\psi\lambda}I_{\lambda\lambda}^{-1}\). Then \(U_{\mathrm{eff}}=U_\psi-AU_\lambda\), so
In the second line, \(I_{\lambda\lambda}\) is symmetric, so \(A^\mathsf{T}=I_{\lambda\lambda}^{-1}I_{\lambda\psi}\); substituting this expression makes the last three terms combine as shown. This calculation also explains why using \(I_{\psi\psi}\) alone can give the wrong scale: it counts variation in the score that is shared with nuisance estimation.
Calibration and Conditions
For independent, identically distributed observations in a regular parametric model, the scalar score statistic has an asymptotic chi-square calibration under the null. In particular, under \(H_0\), \(S\) converges in distribution to a chi-square distribution with one degree of freedom. A two-sided test at level \(\alpha\) rejects for large values of \(S\), using the upper \(\alpha\) quantile of that distribution.
This statement requires the usual regular likelihood conditions: the true parameter is an interior point of the parameter space; the model is differentiable with respect to its parameters; scores have finite, nonsingular Fisher information; and the restricted maximum likelihood estimator of the nuisance parameter is consistent and regular. Also require \(I_{\lambda\lambda}\) and \(I_{\mathrm{eff}}\) to be nonsingular and positive, respectively. The calibration is not automatically valid at a boundary, under non-identifiability, or when regularity conditions fail. The chi-square result is an asymptotic approximation, not a claim that the statistic has that distribution exactly in a finite sample.
The statistic is squared, so it tests for departure in either direction. If the scientific alternative is one-sided, the sign of the efficient score matters and a one-sided procedure needs an appropriate directional calibration; the squared statistic by itself discards that sign.
Worked Calculations
Worked Example: Testing a Bernoulli Success Probability
Suppose \(X_1,\ldots,X_{40}\) are independent Bernoulli observations with success probability \(p\), and 25 are successes. Test \(H_0:p=0.5\) using the score test.
The log-likelihood, apart from no parameter-dependent terms, is \(\ell(p)=25\log p+15\log(1-p)\). Its score is \(U(p)=25/p-15/(1-p)\), and its Fisher information is \(I(p)=40/[p(1-p)]\). There is no nuisance parameter, so evaluate both directly at the null:
Thus \(S=20^2/160=2.5\). Equivalently, the score is \(25-40(0.5)=5\) divided by the null success probability variance \(40(0.5)(0.5)=10\), giving \(5^2/10=2.5\). Under the asymptotic chi-square calibration, the two-sided p-value is about \(0.114\). This calculation uses the null value \(0.5\), not the unrestricted estimate \(25/40\).
Worked Example: Testing a Normal Mean With Unknown Variance
Suppose \(X_1,\ldots,X_5\) are independent \(N(\mu,\sigma^2)\) observations with observed values \(1,2,3,4,5\). Test \(H_0:\mu=2.5\), treating \(\sigma^2\) as a nuisance parameter.
Under the null, the restricted maximum likelihood estimate of the variance is the average squared deviation from 2.5. The squared deviations sum to \(2.25+0.25+0.25+2.25+6.25=11.25\), so \(\tilde{\sigma}^2=11.25/5=2.25\). The mean score is \(U_\mu=\sum_i(X_i-\mu)/\sigma^2\); at the restricted fit it equals \((15-5(2.5))/2.25=2.5/2.25=10/9\).
For a normal model, the expected information for \(\mu\) is \(5/\sigma^2\). The cross-information between \(\mu\) and \(\sigma^2\) is zero, so the nuisance adjustment does not change this information. Consequently,
The result is an asymptotic score statistic, not the exact finite-sample \(t\)-statistic. The restricted fit is straightforward: set the mean to its null value and estimate only the variance under that constraint.
Worked Example: Testing a Logistic Slope With a Restricted Intercept Fit
Consider a logistic model with \(P(Y=1\mid x)=\operatorname{logit}^{-1}(\alpha+\beta x)\), where \(x=0\) for 10 observations and \(x=1\) for 10 observations. There are 3 successes in the \(x=0\) group and 7 in the \(x=1\) group. Test \(H_0:\beta=0\), with intercept \(\alpha\) as a nuisance parameter.
Under the null, both groups share a success probability. The restricted estimate is the overall proportion, \(\tilde p=(3+7)/20=0.5\), so \(\tilde\alpha=\operatorname{logit}(0.5)=0\). The slope score is \(\sum_i x_i(Y_i-\tilde p)\). Only the \(x=1\) observations contribute, giving \(U_\beta=7-10(0.5)=2\). The intercept score is the overall residual total, \(U_\alpha=(3+7)-20(0.5)=0\); hence the restricted fit has zero nuisance score.
For logistic regression, the expected information is the weighted design cross-product. At the restricted fit every weight is \(\tilde p(1-\tilde p)=0.25\). Since \(\sum 1=20\), \(\sum x=10\), and \(\sum x^2=10\), the information for \((\alpha,\beta)\) is
The efficient information for the slope is \(I_{\beta\beta}-I_{\beta\alpha}I_{\alpha\alpha}^{-1}I_{\alpha\beta}=2.5-(2.5)(1/5)(2.5)=1.25\). Thus \(S=2^2/1.25=3.2\), with an asymptotic chi-square-one p-value of about \(0.074\). The restricted model requires only the pooled success proportion; no unrestricted logistic fit is needed.
Common Mistake: Ignoring Nuisance-Parameter Adjustment
A tempting shortcut is to use \(U_\psi^2/I_{\psi\psi}\) whenever the restricted nuisance score is zero. The zero score justifies simplifying the numerator; it does not justify ignoring nuisance parameters in the information. In the logistic example, using \(I_{\beta\beta}=2.5\) alone would give \(4/2.5=1.6\), rather than the correct \(3.2\). The intercept and slope scores are correlated, so only \(1.25\) units of information remain for the slope after accounting for the intercept.
A second error is to calculate the score at the unrestricted estimate. The score test is defined at the restricted fit: impose \(\psi=\psi_0\), estimate nuisance parameters there, and evaluate both score and information at that point. This is exactly what allows the test to avoid fitting the unrestricted model.
Set \(\psi=\psi_0\) and find the restricted nuisance estimate \(\tilde{\lambda}\).
Compute \(U_\psi\), \(U_\lambda\), and the expected information blocks at \((\psi_0,\tilde{\lambda})\).
Form \(U_{\mathrm{eff}}\) and \(I_{\mathrm{eff}}\), then calculate \(S=U_{\mathrm{eff}}^2/I_{\mathrm{eff}}\).
Under regularity conditions, compare \(S\) with a chi-square-one reference distribution; check for boundary or other nonregular features.
Check Your Understanding
Answer each question using the definitions and calculations above.
- Why does an interior restricted maximum make the nuisance-score term in \(U_{\mathrm{eff}}\) vanish?
- What does \(I_{\mathrm{eff}}\) measure, and why can it be smaller than \(I_{\psi\psi}\)?
- In the logistic example, what incorrect score statistic results from using \(I_{\beta\beta}\) without nuisance adjustment?
- Why is the Bernoulli score evaluated at \(p=0.5\), rather than at the observed success proportion?
- Name two circumstances in which the usual chi-square-one calibration may fail.