When a Wald Calculation Deserves a Closer Look
A Wald statistic compresses an estimate and its estimated standard error into a single standardized distance from the null value. This is effective when the likelihood is locally well approximated by a quadratic and the estimate is well inside the parameter space. With weak information, the likelihood can be flat or distinctly asymmetric; near a boundary, the estimate or its standard error can behave irregularly. In these settings, the arithmetic of a Wald statistic may be correct while its usual large-sample interpretation is a poor guide.
A useful diagnostic is to compare the Wald calculation with the likelihood profile: how quickly does the log-likelihood fall as the parameter moves away from its maximum? Also check whether the conclusion changes substantially after a smooth change of parameter scale. These checks do not guarantee that another procedure is calibrated, but they reveal when a symmetric, local approximation is doing too much work.
The profile likelihood keeps the parameter of interest fixed at each candidate value and maximizes over nuisance parameters. Its loss from the maximum is \(\ell(\hat{\theta})-\ell(\theta_0)\); twice this loss is the likelihood-ratio deviance for the scalar restriction. Unlike a Wald statistic, this loss reads the likelihood away from its maximum rather than summarizing only local curvature at the maximum.
Three Worked Diagnostics
Worked Example: One Success in a Small Binomial Sample
Suppose \(X\sim\operatorname{Binomial}(10,p)\), and the observed count is \(X=1\). Test \(H_0:p=0.2\) using a plug-in Wald statistic and compare it with the profile-likelihood loss.
The maximum-likelihood estimate is \(\hat p=1/10=0.1\). The estimated standard error is \(\sqrt{\hat p(1-\hat p)/10}=\sqrt{0.09/10}=0.094868\). Thus \(W=(0.1-0.2)/0.094868\approx-1.0541\), and \(W^2\approx1.1111\).
Ignoring the binomial coefficient, the log-likelihood is \(\ell(p)=\log(p)+9\log(1-p)\). The loss at the null value is
The deviance is therefore \(2(0.366900)=0.733800\), and the signed square root, with sign \(\operatorname{sign}(\hat p-0.2)\), is approximately \(-0.85662\). The Wald and profile calculations differ appreciably here. With only one success, the likelihood is not well summarized by a symmetric quadratic over the distance from \(0.1\) to \(0.2\). This comparison diagnoses approximation sensitivity; by itself, it does not establish which test has the desired calibration in every setting.
Worked Example: A Wald Statistic on Two Parameter Scales
Let \(Y\sim\operatorname{Poisson}(10\lambda)\), observe \(Y=1\), and test \(H_0:\lambda=0.2\). Compare the Wald statistic for \(\lambda\) with the one for \(\eta=\log\lambda\).
The estimate is \(\hat\lambda=Y/10=0.1\). Its plug-in standard error is \(\sqrt{\hat\lambda/10}=\sqrt{0.01}=0.1\), giving \(W_\lambda=(0.1-0.2)/0.1=-1\). For the log scale, \(\hat\eta=\log(0.1)\). The delta-method standard error is \(0.1/0.1=1\), so \(W_\eta=(\log(0.1)-\log(0.2))/1=\log(0.5)\approx-0.6931\). Their squared values are \(1\) and approximately \(0.4805\). The finite-sample Wald results differ substantially.
The log-likelihood, up to a constant, is \(\ell(\lambda)=\log(\lambda)-10\lambda\) for this observation. Its loss at the null is \(\ell(0.1)-\ell(0.2)=\log(0.5)+1\approx0.306853\). The deviance is about \(0.613706\), with signed square root approximately \(-0.7834\). The likelihood loss offers a third calculation, based on the shape between the estimate and null. The scale comparison is a warning that neither Wald number should be treated as an intrinsic finite-sample distance.
Worked Example: Zero Counts at a Boundary
Suppose \(Y\sim\operatorname{Poisson}(20\lambda)\), with \(\lambda\geq0\), and observe \(Y=0\). Consider a Wald calculation for testing \(\lambda=0.05\).
The likelihood is proportional to \(\exp(-20\lambda)\), so it is maximized at the boundary \(\hat\lambda=0\). The usual plug-in standard error formula gives \(\sqrt{\hat\lambda/20}=0\). The proposed Wald statistic would divide \(0-0.05\) by zero, so it is undefined. There is no finite Wald statistic to interpret.
The profile log-likelihood, up to a constant, is \(\ell(\lambda)=-20\lambda\). Its loss at \(0.05\) is \(1\), and twice that loss is \(2\). This finite likelihood comparison does not make the usual interior, chi-square Wald approximation valid: the estimate is on the boundary and the plug-in curvature-based standard error has degenerated. Boundary problems require separate calibration considerations.
Why Curvature Can Mislead
The next calculation makes precise the link between a Wald distance and a profile-likelihood distance. It also identifies a condition under which their agreement can fail. Treat \(\ell\) as a three-times continuously differentiable scalar log-likelihood on an interval containing an interior maximizer \(\hat\theta\) and a candidate \(\theta_0\). Suppose \(\ell'(\hat\theta)=0\), \(J=-\ell''(\hat\theta)>0\), and \(|\ell'''(\theta)|\leq M\) throughout the interval between these two values.
Proof. Taylor's theorem about \(\hat\theta\), with a remainder evaluated at some point \(\xi\) between \(\hat\theta\) and \(\theta_0\), gives \(\ell(\theta_0)=\ell(\hat\theta)+\ell'(\hat\theta)\delta+\ell''(\hat\theta)\delta^2/2+\ell'''(\xi)\delta^3/6\). The first derivative term is zero, and \(\ell''(\hat\theta)=-J\). Rearranging yields \(R=-\ell'''(\xi)\delta^3/6\), so the derivative bound gives the stated bound on \(R\). Multiplying the loss expansion by two gives the bound on \(D\). \(\square\)
The leading term \(J\delta^2\) is exactly the squared Wald distance when the standard error is \(J^{-1/2}\). But the error bound is informative only relative to that leading term: for \(\delta\ne0\), the relative bound is \(M|\delta|/(3J)\). If curvature \(J\) is small, or the third derivative is large, this ratio need not be small. In a flat or skewed profile, a standard error based on curvature at the maximum cannot by itself ensure a good approximation at the null value.
What a Smooth Change of Scale Does—and Does Not—Fix
Wald statistics are not exactly invariant to reparameterization. The following first-order fact explains why differences can shrink in a regular local setting without disappearing in a small sample. Let \(g\) be continuously differentiable and strictly monotone near an interior true value \(\theta_0\), with \(g'(\theta_0)\ne0\). Suppose \(\hat\theta\) converges in probability to \(\theta_0\), and use the delta-method standard error \(|g'(\hat\theta)|s\) for \(g(\hat\theta)\), where \(s\) is the standard error used for \(\hat\theta\).
Proof. By the mean value theorem, for some \(\tilde\theta\) between \(\hat\theta\) and \(\theta_0\), \(g(\hat\theta)-g(\theta_0)=g'(\tilde\theta)(\hat\theta-\theta_0)\). Taking absolute values and dividing by the transformed standard error gives \[ \frac{|g(\hat\theta)-g(\theta_0)|/(|g'(\hat\theta)|s)} {|\hat\theta-\theta_0|/s} =\frac{|g'(\tilde\theta)|}{|g'(\hat\theta)|}. \] Both \(\tilde\theta\) and \(\hat\theta\) approach \(\theta_0\); continuity and \(g'(\theta_0)\ne0\) make the ratio tend to one. The sign statement follows from whether \(g\) is increasing or decreasing. This local result does not assert equality at finite sample sizes, as the Poisson example demonstrates. \(\square\)
A Practical Diagnostic
When a Wald result looks surprising, inspect the ingredients rather than relying only on its reported statistic or \(p\)-value. The following sequence uses standard likelihood and standard-error calculations without treating any one diagnostic as an automatic replacement for calibration.
Determine whether the estimate is interior and whether the null value is far from it relative to the estimated standard error.
Compare the log-likelihood loss on either side of its maximum. Strong asymmetry or a flat region makes a single curvature-based standard error less descriptive.
For a positive parameter, compare the calculation on the original and log scales, taking care to use the corresponding transformed standard error.
At a boundary, or when a plug-in standard error is zero or undefined, do not report a finite Wald statistic by silently modifying its denominator.
A common mistake is to assume that a large estimated standard error automatically makes a Wald test conservative, or that a small one makes it precise. The standard error describes estimated local spread under the model; it does not certify that the likelihood is symmetric over the relevant distance, that the estimate is away from a boundary, or that a finite-sample normal approximation is adequate. Likewise, agreement between Wald and profile calculations is a useful check, not a proof of correct size.
These diagnostics matter because the Wald procedure is attractive precisely for its simplicity: estimate, standardize, and compare with a reference distribution. Weak information and boundaries expose what that simplicity assumes. Reading the profile and checking the scale make those assumptions visible, helping distinguish a well-behaved local approximation from a fragile one.
Check Your Understanding
Use the definitions and calculations above to answer these questions.
- Why does the observed information determine a curvature-based standard error, and what feature of the likelihood does that standard error fail to describe by itself?
- In the binomial example, what are the profile deviance and its signed square root, and why do they differ from the Wald values?
- For the Poisson example, why are the Wald statistics on the rate and log-rate scales not exactly equal?
- What happens to the plug-in Wald statistic in the zero-count boundary example, and why is it not valid to report a finite value?
- State one condition from the Taylor expansion result that helps the likelihood-ratio deviance approximate the squared Wald distance.