Wald, Score, and Likelihood-Ratio Equivalence

Three Routes to Testing the Same Hypothesis

For a regular nested hypothesis, three familiar tests ask how strongly the data disagree with the null in different ways. The Wald test measures how far the unrestricted estimate lies from the null. The score test measures the likelihood’s slope at the restricted estimate. The likelihood-ratio test compares the best likelihood available under the null with the unrestricted maximum. Their calculations can look different, but locally they are driven by the same quadratic shape of the log-likelihood.

The equivalence is asymptotic, not an assertion that the statistics are equal for a particular sample. We derive their common limit and then calculate examples where the finite-sample values either coincide or differ. The distinction matters in applications: using a chi-square reference distribution does not erase the effects of sample size, skewness, or nuisance-parameter estimation.

Setup and Regularity Conditions

Let \(X_1,\ldots,X_n\) be independent observations from a correctly specified regular parametric model with log-likelihood \(\ell_n(\theta)\), where \(\theta=(\psi^T,\eta^T)^T\). The parameter of interest \(\psi\) has dimension \(q\), and \(\eta\) is a nuisance parameter. We test the nested hypotheses \(H_0:\psi=\psi_0\) against the unrestricted alternative. The true parameter under the null is \((\psi_0,\eta_0)\).

Write the per-observation Fisher information at the null in blocks:

Assume \(I_{\eta\eta}\) is nonsingular. The efficient information for \(\psi\), after accounting for the nuisance parameter, is the Schur complement \(J=I_{\psi\psi}-I_{\psi\eta}I_{\eta\eta}^{-1}I_{\eta\psi}\), which is positive definite under the stated conditions. This is per-observation information; the corresponding information in \(n\) observations is \(nJ\).

At the restricted maximum, the nuisance score is zero. The score statistic therefore uses the score in the tested directions and the total efficient information \(n\hat J_n\), not just per-observation information. Equivalently, \(S_n=(U_\psi/\sqrt n)^T\hat J_n^{-1}(U_\psi/\sqrt n)\).

Why the Statistics Have the Same Limit

The key is the local likelihood after nuisance parameters have been adjusted to their best-fitting values. For bounded \(h\), consider changing \(\psi\) from \(\psi_0\) to \(\psi_0+h/\sqrt n\), while maximizing over the nuisance parameter. The regular likelihood expansion gives

uniformly over bounded \(h\). Here \(\ell_{p,n}\) denotes the likelihood with the nuisance parameter locally optimized, and \(\Delta_n=U_\psi/\sqrt n\) at the restricted fit. Under the null, \(\Delta_n\xrightarrow{d}N(0,J)\). In particular, the nuisance adjustment is essential: the covariance of this adjusted score is \(J\), not generally \(I_{\psi\psi}\).

Derivation. The quadratic expression in the local expansion is maximized at \(h=J^{-1}\Delta_n\). Its maximum is \(\frac12\Delta_n^TJ^{-1}\Delta_n\). The unrestricted estimate has local displacement \(\sqrt n(\hat\psi-\psi_0)=J^{-1}\Delta_n+o_p(1)\), as follows from the local score equation. Thus the likelihood-ratio statistic is

For the score statistic, \(U_\psi/\sqrt n=\Delta_n\), and consistency gives \(\hat J_n\xrightarrow{p}J\). Hence \(S_n=\Delta_n^TJ^{-1}\Delta_n+o_p(1)\). For the Wald statistic, substitute the local displacement of the unrestricted estimate and the same consistent information estimate:

Finally, write \(Z_n=J^{-1/2}\Delta_n\). Since \(\Delta_n\xrightarrow{d}N(0,J)\), \(Z_n\xrightarrow{d}N(0,I_q)\), and \(\Delta_n^TJ^{-1}\Delta_n=Z_n^TZ_n\xrightarrow{d}\chi_q^2\). Slutsky’s theorem completes the argument for all three statistics. \(\square\)

This derivation also shows what “equivalence” means: each statistic has the same leading random term under the null, so their differences are \(o_p(1)\). It does not say they have the same value at finite \(n\). Under local alternatives \(\psi=\psi_0+a/\sqrt n\), the same quadratic argument gives a noncentral chi-square limit with noncentrality parameter \(a^TJa\); under the null, \(a=0\), yielding the central chi-square result.

Worked Comparisons

Interpretation and a Common Pitfall

The tests have different practical constructions. Wald is convenient when the unrestricted estimate and its covariance are readily available. The score test needs a fit under the null, but not an unrestricted fit. The likelihood-ratio test requires both fits and compares their likelihoods. Under the regular null and large samples, all use the same local information about the \(q\) tested directions, so the degrees of freedom are \(q\).

Common mistake: mixing total and per-observation information. If \(U_\psi\) is the score from all \(n\) observations, dividing its squared size by per-observation information alone makes the score statistic too large by a factor of \(n\). In the binomial example, \(U_p=12\) and per-observation information is \(4\), but the denominator must be total information \(10(4)=40\), giving \(144/40=3.6\). The incorrect calculation \(144/4=36\) does not have the stated chi-square limit. Equivalently, divide the total score by \(\sqrt n\) and use per-observation information.

The regularity conditions are substantive. A parameter on the boundary, a singular information matrix, or a non-identifiable model can invalidate the ordinary chi-square conclusion. In regular problems, finite-sample discrepancies remain possible; the approximation improves asymptotically, but it is not an algebraic identity among the tests.