Local Equivalence of Wald, Score, and Likelihood-Ratio Tests

Why Three Tests Can Agree

Wald, score, and likelihood-ratio tests approach a null hypothesis from different directions. A Wald test measures how far the unrestricted estimate lies from the null; a score test measures the slope of the log likelihood at the null; and a likelihood-ratio test compares the maximized log likelihoods. In finite samples, their statistics need not match. Under regularity conditions, however, the log likelihood is locally quadratic, and all three statistics reduce to the same quadratic form.

The result also applies to alternatives that approach the null at rate \(n^{-1/2}\). Such alternatives are close enough to be difficult to distinguish from the null in a large sample, but their effect remains visible in the limiting distribution. That makes local equivalence more informative than a statement about the null alone.

Local Likelihood Setup

Use a \(q\)-dimensional parameter \(\eta\) to represent the tested directions, with null value \(\eta=0\). If the model also has nuisance parameters, take \(\ell_n(\eta)\) below to be the profile log likelihood for these tested directions. In that case, the information matrix \(I\) is the information for the profile likelihood, including the effect of nuisance estimation. Assume the null is an interior point.

The regularity conditions used here are: the parameter is interior; the likelihood is sufficiently differentiable near the null; \(J_n(0)/n\xrightarrow{p}I\); and the unrestricted maximum-likelihood estimator \(\hat\eta\) is consistent and localized at rate \(n^{-1/2}\), so \(\sqrt n\hat\eta=O_p(1)\). The following two local expansions are required, uniformly for \(h\) in each fixed bounded set:

The score expansion is stated separately on purpose. A small remainder in a log-likelihood expansion does not, by itself, imply a small remainder after differentiating it. The score expansion follows, for example, if the score is differentiable and its derivative satisfies a suitable uniform convergence condition in shrinking neighborhoods of the null. We assume the expansion directly here, so the argument does not rely on differentiating an unspecified remainder.

Deriving the Three Statistics

The local quadratic expression \(h^\mathsf{T}Z_n-\tfrac12h^\mathsf{T}Ih\) is maximized at \(h=I^{-1}Z_n\). The unrestricted estimator has this same first-order local representation. In particular, the score expansion, the likelihood equation \(U_n(\hat\eta)=0\), and localization give

To see the role of the explicit score condition, put \(\hat h=\sqrt n\hat\eta\). On events where \(\hat h\) is bounded, the uniform score expansion gives \(0/\sqrt n=Z_n-I\hat h+o_p(1)\). Since \(\hat h=O_p(1)\), the probability of being outside a sufficiently large bounded set can be made arbitrarily small. Solving the displayed equation on those sets yields the representation above.

For the score statistic, use the score at the null and the null information:

The last equality uses \(J_n(0)/n\xrightarrow{p}I\) and \(Z_n=O_p(1)\). The latter follows under the null from the usual score central limit theorem, and under the local alternatives considered below from the corresponding shifted limit.

For the Wald statistic, use the unrestricted estimate and its information matrix:

This form covers use of observed or expected information, provided the information per observation evaluated at the unrestricted estimate converges in probability to \(I\). The likelihood-ratio statistic is \(R_n=2\{\ell_n(\hat\eta)-\ell_n(0)\}\). By localization, the uniform local likelihood expansion can be applied at \(\hat h=\sqrt n\hat\eta\). Maximizing its quadratic part gives

These arguments establish the theorem: each statistic has the same leading quadratic form, so subtracting any two leaves only an \(o_p(1)\) difference. \(\square\)

Null and Contiguous-Alternative Limits

Under the null, assume the usual score central limit theorem \(Z_n\xrightarrow{d}N(0,I)\). It follows that \(I^{-1/2}Z_n\xrightarrow{d}N(0,\mathrm{Id}_q)\), where \(\mathrm{Id}_q\) is the \(q\)-dimensional identity matrix. Thus the common quadratic form converges to a chi-square distribution with \(q\) degrees of freedom.

Now consider a sequence of local alternatives \(\eta_n=h/\sqrt n\), with fixed \(h\). Under the standard local asymptotic normality conditions, the likelihood ratio between this sequence and the null has the local expansion above. The change-of-measure result known as Le Cam’s third lemma then gives \(Z_n\xrightarrow{d}N(Ih,I)\) under the local alternatives. Therefore:

Here \(\chi^2_q(\lambda)\) denotes the noncentral chi-square distribution with \(q\) degrees of freedom and noncentrality parameter \(\lambda\). The three statistics consequently have the same null limit and the same local-alternative limit. This does not say their finite-sample values are equal; it says their differences become negligible in probability under these conditions.

Worked Examples

Interpretation and Common Pitfall

The equivalence explains why the three tests often yield similar conclusions in large, regular GLM samples. Under the null, their statistics have the same limiting reference distribution; under \(n^{-1/2}\)-local alternatives, they also share the same limiting noncentrality. Since the chi-square limit has a continuous distribution, their asymptotic p-values agree as well, subject to the stated assumptions.

Local equivalence can fail outside regular settings. The derivation assumes an interior null, nonsingular information, suitable differentiability, and a root-\(n\) localized estimator. Boundary parameters, nonidentifiability, sparse data, or an ill-conditioned information matrix can invalidate those assumptions. Even when the assumptions hold asymptotically, meaningful finite-sample differences remain possible, as the logistic and Poisson calculations illustrate.