Local Power of Wald, Score, and Likelihood-Ratio Tests

Why Local Power Matters

Two tests can have the same limiting null distribution and still differ in finite samples. To compare their behavior more closely, consider alternatives that move toward the null as the sample size grows. Alternatives separated from the null by order \(n^{-1/2}\) are called local alternatives: they are close enough to make large-sample approximations relevant, but far enough to retain nontrivial power in the limit.

For regular models, Wald, score, and likelihood-ratio tests have the same first-order power against these alternatives. Their common limiting power depends on how much Fisher information the data contain in the direction that violates the restriction. This tutorial derives that result and works through several calculations.

Local Alternatives and the Information in a Restriction

Definition: Let \(\theta\) be a parameter in an open subset of \(\mathbb{R}^p\), and let \(\theta_0\) satisfy the null hypothesis. A sequence \(\theta_n=\theta_0+h/\sqrt{n}\), for a fixed vector \(h\in\mathbb{R}^p\), is a local alternative in direction \(h\). The vector \(h\) describes the direction and size of the departure; the actual parameter difference shrinks to zero as \(n\) increases.

Suppose the null imposes \(q\) smooth restrictions \(g(\theta)=0\), where \(g\) maps to \(\mathbb{R}^q\), \(g(\theta_0)=0\), and its derivative matrix \(A=\nabla g(\theta_0)\) has rank \(q\). The first-order departure from the null under \(\theta_n\) is \(Ah/\sqrt n\). Thus a direction \(h\) tangent to the null, for which \(Ah=0\), does not violate the restriction to first order.

Write \(I=I(\theta_0)\) for the per-observation Fisher information matrix. In a regular model, \(I\) is positive definite. The quantity

\lambda(h)=(Ah)^\mathsf{T}\left(AI^{-1}A^\mathsf{T}\right)^{-1}Ah

is the noncentrality parameter for the local test. It is nonnegative and equals zero precisely when \(Ah=0\). It measures the local displacement in the metric determined by the sampling uncertainty of the estimated restriction.

Theorem: Consider independent, identically distributed observations from a finite-dimensional regular model, with \(\theta_0\) an interior point and nonsingular Fisher information \(I(\theta_0)\). Suppose the model is locally asymptotically normal at \(\theta_0\), the maximum likelihood estimator is asymptotically linear and consistent, and the constrained and unconstrained likelihoods admit the usual quadratic local expansions. For testing \(g(\theta)=0\), with \(g(\theta_0)=0\) and \(\operatorname{rank}(A)=q\), all three standard statistics—the Wald, score, and likelihood-ratio statistics—converge under \(\theta_n=\theta_0+h/\sqrt n\) to a noncentral chi-square distribution with \(q\) degrees of freedom and noncentrality parameter \(\lambda(h)\). Consequently, for a common asymptotic critical value \(c\), each test's limiting rejection probability is \(P\{\chi^2_q(\lambda(h))>c\}\).

Here local asymptotic normality means that for fixed \(h\), the log-likelihood ratio has the expansion

\ell_n(\theta_0+h/\sqrt n)-\ell_n(\theta_0) =h^\mathsf{T}\Delta_n-\frac12 h^\mathsf{T}Ih+o_{P_{\theta_0}}(1), \qquad \Delta_n\xrightarrow{d}N(0,I)

under the null. The stated regularity conditions rule out such complications as a singular information matrix or a parameter on the boundary. They also ensure that the same local quadratic approximation applies to the three statistics.

Why the Three Statistics Have the Same Local Limit

The key fact for the Wald statistic is that the maximum likelihood estimator satisfies, under the local alternative,

\sqrt n(\hat\theta-\theta_0)\xrightarrow{d}N(h,I^{-1}).

For a smooth restriction, a first-order Taylor expansion gives \(\sqrt n\,g(\hat\theta)=A\sqrt n(\hat\theta-\theta_0)+o_p(1)\). The covariance matrix of this restricted estimator is \(AI^{-1}A^\mathsf{T}\). Therefore, the Wald statistic has the same limit as

\left\{A\sqrt n(\hat\theta-\theta_0)\right\}^{\mathsf{T}} \left(AI^{-1}A^\mathsf{T}\right)^{-1} \left\{A\sqrt n(\hat\theta-\theta_0)\right\}.

The vector inside the quadratic form converges to a normal vector with mean \(Ah\) and covariance \(AI^{-1}A^\mathsf{T}\). A quadratic form of this type has a noncentral chi-square distribution, with degrees of freedom \(q\) and noncentrality parameter \(\lambda(h)\).

For the score and likelihood-ratio statistics, it is useful to describe the same argument in coordinates that separate the tested parameter from nuisance parameters. Write \(\theta=(\psi,\eta)\), with \(\psi\in\mathbb{R}^q\), and test \(\psi=\psi_0\). Partition the information matrix conformably, and define the efficient information for \(\psi\), after accounting for the nuisance parameter \(\eta\), as

J=I_{\psi\psi}-I_{\psi\eta}I_{\eta\eta}^{-1}I_{\eta\psi}.

Under a local alternative \(\psi_n=\psi_0+a/\sqrt n\), \(\eta_n=\eta_0+b/\sqrt n\), the efficient score has limiting distribution \(N(Ja,J)\). Its nuisance component has been removed, so the local shift \(b\) does not appear. The score statistic is the efficient score's quadratic form using \(J^{-1}\); it therefore converges to a noncentral chi-square distribution with noncentrality \(a^\mathsf{T}Ja\).

The quadratic local log-likelihood for the tested coordinate, after profiling out \(\eta\), has the form

u^\mathsf{T}\Delta_{\mathrm{eff}}-\frac12u^\mathsf{T}Ju+o_p(1),

where \(u=\sqrt n(\psi-\psi_0)\) and \(\Delta_{\mathrm{eff}}\) is the efficient score. Maximizing this expression over \(u\) gives \(\frac12\Delta_{\mathrm{eff}}^\mathsf{T}J^{-1}\Delta_{\mathrm{eff}}\); imposing the null sets \(u=0\), giving zero. Twice the difference, the likelihood-ratio statistic, is therefore \(\Delta_{\mathrm{eff}}^\mathsf{T}J^{-1}\Delta_{\mathrm{eff}}+o_p(1)\). It has the same limit as the score statistic. The Wald statistic has this limit as well, since \(J^{-1}\) is the asymptotic covariance matrix of \(\sqrt n(\hat\psi-\psi_0)\).

Result: For coordinate restrictions \(\psi=\psi_0\), all three tests have noncentrality parameter \(a^\mathsf{T}Ja\). For general smooth restrictions, this is the same information-based quantity written in the original parameter coordinates as \((Ah)^\mathsf{T}(AI^{-1}A^\mathsf{T})^{-1}Ah\).

Worked Examples

Worked Example: A Local Alternative for a Bernoulli Probability

Let \(X_1,\ldots,X_n\) be independent Bernoulli observations with success probability \(p\). Test \(H_0:p=1/2\) against a two-sided alternative. Consider \(p_n=1/2+h/\sqrt n\), where \(h=1\).

For one Bernoulli observation, the Fisher information for \(p\) is \(I(p)=1/\{p(1-p)\}\). At \(p_0=1/2\), \(I(p_0)=4\). Here \(A=1\), so

\lambda=\frac{h^2}{I(p_0)^{-1}}=h^2 I(p_0)=4.

For example, at \(n=400\), the local alternative is \(p_n=1/2+1/20=0.55\). The limiting standardized statistic has a unit-variance normal distribution shifted by \(\sqrt{\lambda}=2\); the corresponding two-sided chi-square test rejects when its absolute value exceeds approximately \(1.96\). Its limiting power is

P(|Z+2|>1.96) =\Phi(0.04)+\Phi(-3.96) \approx 0.5160, \qquad Z\sim N(0,1).

Thus the three tests have the same first-order power against this sequence of alternatives. The calculation concerns their shared limit, not a claim that their finite-sample rejection probabilities are exactly equal.

Worked Example: Testing a Difference Between Two Means

Suppose \(X_1,\ldots,X_n\) are independent bivariate normal observations with mean vector \(\mu=(\mu_1,\mu_2)^\mathsf{T}\) and known covariance matrix \(\Sigma=\begin{pmatrix}1&0.5\\0.5&1\end{pmatrix}\). Test \(H_0:\mu_1-\mu_2=0\). Take \(\mu_0=(0,0)^\mathsf{T}\), and consider the local alternative \(\mu_n=(1/\sqrt n,0)^\mathsf{T}\).

The information for the mean vector in one observation is \(I=\Sigma^{-1}\), so \(I^{-1}=\Sigma\). The restriction has derivative \(A=(1,-1)\), and the local direction is \(h=(1,0)^\mathsf{T}\). Hence \(Ah=1\), while

AI^{-1}A^\mathsf{T} =(1,-1) \begin{pmatrix}1&0.5\\0.5&1\end{pmatrix} \begin{pmatrix}1\\-1\end{pmatrix} =1+1-0.5-0.5=1.

Therefore \(\lambda=(Ah)^2/(AI^{-1}A^\mathsf{T})=1\), with one restriction and thus one degree of freedom. Although the two observations within each vector are positively correlated, the variance of their difference is \(1\); that is precisely the variance appearing in this restriction's information calculation.

Worked Example: A Tested Mean With an Unknown Nuisance Mean

Let each observation be bivariate normal with mean \((\psi,\eta)^\mathsf{T}\) and known covariance \(\Sigma=\begin{pmatrix}1&0.6\\0.6&1\end{pmatrix}\). Test \(H_0:\psi=0\), treating \(\eta\) as an unrestricted nuisance parameter. Consider local alternatives \(\psi_n=a/\sqrt n\) and \(\eta_n=\eta_0+b/\sqrt n\).

The information per observation is

I=\Sigma^{-1} =\frac{1}{1-0.6^2} \begin{pmatrix}1&-0.6\\-0.6&1\end{pmatrix}.

Writing \(\psi\) first, the efficient information for \(\psi\) is the Schur complement

J=I_{\psi\psi}-I_{\psi\eta}I_{\eta\eta}^{-1}I_{\eta\psi} =\frac{1}{0.64}-\frac{(-0.6/0.64)^2}{1/0.64} =1.5625-0.5625=1.

Thus \(\lambda=a^2J=a^2\), regardless of \(b\). If \(\eta\) were known, the information for \(\psi\) would instead be \(I_{\psi\psi}=1.5625\). With \(\eta\) unknown, the correlated second coordinate cannot add information about \(\psi\) without also informing the nuisance mean. The efficient-information calculation accounts for that loss.

What the Result Does—and Does Not—Say

The noncentrality parameter explains how local power changes with the alternative. A larger displacement in a direction measured precisely by the data increases \(\lambda\), and hence increases the limiting rejection probability for a fixed critical value. A displacement tangent to the null has \(Ah=0\), so its noncentrality parameter is zero: these tests have no first-order power advantage against that local sequence.

A common mistake is to compare local alternatives only by their Euclidean distance from the null. For instance, one might reason that two alternatives with the same value of \(\lVert h\rVert\) must produce the same power. That is generally false. Correction: power depends on the alternative's direction relative to the restriction and on Fisher information, through \(\lambda(h)\), not on Euclidean distance alone. Equal-sized changes in two parameter directions can have very different information and hence different local power.

The result is also a first-order, large-sample comparison. It does not say the statistics are identical in a finite sample, nor does it settle which is preferable when regularity conditions fail. Its value is that it identifies a common local benchmark: under a regular model, choosing among these three tests does not change their limiting power against \(n^{-1/2}\)-scale alternatives.

Check Your Understanding

Use the information-based formulation to answer these questions.

For a local alternative \(\theta_n=\theta_0+h/\sqrt n\), what does the vector \(h\) represent, and why does the parameter difference shrink with \(n\)?
For a smooth restriction with derivative matrix \(A\), what condition on \(h\) makes the noncentrality parameter zero?
In a model with tested parameter \(\psi\) and nuisance parameter \(\eta\), why does the efficient information, rather than \(I_{\psi\psi}\) alone, determine local power?
For the Bernoulli example with \(p_0=1/2\) and \(h=1\), calculate the noncentrality parameter.
Why need not two local alternatives at the same Euclidean distance from the null have the same limiting power?