Local Equivalence of the Three Classical Tests

Why Compare the Three Tests Locally?

For a regular hypothesis in a linear model, likelihood-ratio, score, and Wald tests are built in different ways: the likelihood-ratio test compares optimized likelihoods, the score test uses the slope of the likelihood at the restricted fit, and the Wald test measures the distance from the unrestricted estimate to the null. Their finite-sample formulas need not agree. Yet near the null, all three are governed by the same local curvature of the likelihood. This tutorial derives that first-order equivalence.

The relevant alternatives approach the null as the sample size grows. They are close enough that their distributions cannot be separated from the null by a fixed parameter gap, but far enough to retain nontrivial power. This local comparison is useful for understanding why the tests often have similar large-sample behavior without claiming that their finite-sample statistics are identical.

Local Likelihood Coordinates and Nuisance Parameters

Let \(\gamma\in\mathbb{R}^k\) denote the tested parameter, with null hypothesis \(\gamma=0\), and let \(\lambda\) denote nuisance parameters. The nuisance parameters are not fixed by the hypothesis: they are estimated under both the null and alternative. Write \(\ell_n(\gamma,\lambda)\) for the log-likelihood. Profiling means maximizing over \(\lambda\) for each fixed \(\gamma\); the resulting function is denoted by \(\ell_{p,n}(\gamma)\).

The information relevant to testing \(\gamma\) is not generally the \(\gamma\)-block of the full information matrix by itself. Nuisance estimation reduces the information available for the tested parameter. If the per-observation information matrix at the null is partitioned as \(\mathcal I=\begin{pmatrix}\mathcal I_{\gamma\gamma}&\mathcal I_{\gamma\lambda}\\\mathcal I_{\lambda\gamma}&\mathcal I_{\lambda\lambda}\end{pmatrix}\), the efficient information and efficient score are

Here \(U_{\gamma,n}\) and \(U_{\lambda,n}\) are the respective score blocks at the null. Assume \(\mathcal I_{\lambda\lambda}\) and \(\mathcal I_{\mathrm{eff}}\) are nonsingular. The efficient score subtracts the part of the tested-parameter score that can be explained by movement in nuisance parameters. In what follows, write \(I=\mathcal I_{\mathrm{eff}}\) and \(\Delta_n=n^{-1/2}U_{\mathrm{eff},n}\).

These statements are the local asymptotic normality expansion for the profiled likelihood and its score. Sufficient conditions include a fixed parameter dimension, an interior null point, a likelihood differentiable in quadratic mean, nonsingular information, and a consistent root-\(n\) estimator. In a Gaussian multivariate linear model, one standard setting is fixed response and predictor dimensions, a positive-definite error covariance matrix, and a sequence of full-rank designs for which \(X^\mathsf{T}X/n\) converges to a positive-definite limit. The tested restrictions must have fixed rank and be estimable. The expansion applies in coordinates for those restrictions; nuisance regression coefficients and the error covariance are profiled or estimated as appropriate.

Deriving the Common Quadratic Statistic

The expansion reduces the comparison to a finite-dimensional quadratic function. Its maximizer, the likelihood-ratio statistic, the score statistic, and the Wald statistic can all be calculated directly.

Proof. The quadratic part of the profiled log-likelihood is \(Q_n(h)=h^\mathsf{T}\Delta_n-\tfrac12h^\mathsf{T}Ih\). Since \(I\) is positive definite, this function is strictly concave. Its gradient is \(\Delta_n-Ih\), so its maximizer is \(\tilde h_n=I^{-1}\Delta_n\). Substitution gives

The null sets \(h=0\), for which \(Q_n(0)=0\). Twice the difference between the maximized and null quadratic log-likelihoods is therefore \(\Delta_n^\mathsf{T}I^{-1}\Delta_n\). Uniformity of the likelihood expansion over bounded local parameter sets, together with localization of the likelihood maximizer, makes the actual likelihood-ratio statistic equal to this expression plus \(o_p(1)\).

The efficient score statistic at the null is \(S_n=\Delta_n^\mathsf{T}I^{-1}\Delta_n\), so it has the same first-order expression. The estimator’s local expansion is \(\sqrt n\,\hat\gamma=I^{-1}\Delta_n+o_p(1)\). Consequently the Wald statistic, written using the per-observation efficient information, satisfies

Under the null, \(\Delta_n\xrightarrow{d}N(0,I)\), so \(I^{-1/2}\Delta_n\) converges to a \(k\)-dimensional standard normal vector. Its squared length has a \(\chi^2_k\) distribution. Under \(\gamma=h_0/\sqrt n\), the corresponding limit is \(N(I^{1/2}h_0,I_k)\); the squared length has a noncentral \(\chi^2_k\) distribution with noncentrality \(h_0^\mathsf{T}Ih_0\). Since all three statistics differ from the same quadratic form by \(o_p(1)\), they have the stated common limits. \(\square\)

Worked Examples

Local Power and Practical Interpretation

Under a local alternative \(h_0/\sqrt n\), the limiting noncentrality parameter \(h_0^\mathsf{T}Ih_0\) measures the alternative’s squared distance from the null in the information metric. A shift in a well-informed direction contributes more than an equally sized shift in a poorly informed direction. The common noncentral chi-square limit explains why these tests have the same first-order local power when they use the same hypothesis and information structure.

In multivariate linear models, the hypothesis and error sum-of-squares matrices provide finite-sample representations for classical tests, while the expansion here explains their local relationship in a regular fixed-dimensional regime. The argument does not require deriving a particular MANOVA criterion; it identifies the general likelihood geometry that underlies the local agreement.

Common Mistake: Treating Asymptotic Agreement as Exact Equality

A tempting but incorrect conclusion is: “The three tests have the same quadratic limit, so their observed statistics and p-values must be identical.” The proof only establishes differences of \(o_p(1)\) under the null and fixed contiguous alternatives. Terms beyond the local quadratic approximation can differ, and finite-sample likelihood, score, and Wald statistics can therefore give different numerical values. Their reference distributions and calibrations can also differ in finite samples. The correction is to describe them as first-order asymptotically equivalent, not exactly interchangeable in every sample.