Why Wald and Likelihood-Ratio Tests Disagree in Small Samples

Why Two Tests Can Give Different Answers

A Wald test and a likelihood-ratio test both assess whether a fixed-effect restriction is compatible with the data, but they summarize the fitted likelihood differently. The Wald test uses the estimated coefficient and its estimated standard error. The likelihood-ratio test compares the maximized likelihood under the restriction with the maximized likelihood without it. When information is plentiful and the likelihood is well behaved, these summaries often agree. With limited data, however, the likelihood may be curved unevenly, the estimate may be imprecise, or the large-sample reference distributions may be poor approximations. The resulting p-values can differ enough to fall on opposite sides of a chosen significance level.

The disagreement is not, by itself, evidence that one calculation is wrong. It is a prompt to check the model fits, the coefficient scale, the likelihood shape, and how much independent cluster information supports the test.

Definition: For a scalar fixed-effect coefficient \(\beta\), with null value \(\beta_0\), the Wald statistic is \(W=(\hat{\beta}-\beta_0)^2/SE(\hat{\beta})^2\). For nested models fitted to the same data, the likelihood-ratio statistic is \(G^2=2\{\ell_{\mathrm{full}}-\ell_{\mathrm{null}}\}\), where \(\ell_{\mathrm{full}}\) and \(\ell_{\mathrm{null}}\) are their maximized log likelihoods. Under regularity conditions and \(H_0\), each statistic is approximately chi-square with degrees of freedom equal to the number of tested restrictions.

Conditions for a Meaningful Comparison

Suppose independent clusters are indexed by \(i\), observations within cluster by \(j\), and the response has a GLMM distribution conditional on cluster-specific random effects. The fixed-effect coefficients are collected in \(\boldsymbol{\beta}\). A test of \(H_0:\boldsymbol{R}\boldsymbol{\beta}=\boldsymbol{r}\) imposes \(q\) independent restrictions. For an individual coefficient, \(q=1\). The Wald statistic for a block of restrictions is the estimated distance from the null restriction, measured using the estimated covariance matrix of the coefficients. The likelihood-ratio statistic compares the fitted unrestricted and restricted models.

The usual chi-square calibrations are large-sample approximations. They require, among other things, an identifiable and correctly specified model, a finite interior parameter estimate, and a null restriction that does not put a parameter on the boundary of its allowable values. The likelihood should be sufficiently regular near its maximum. For GLMM fixed-effect inference, the number of independent clusters is often especially important: many observations within a few clusters do not necessarily supply the same information as observations spread across many independent clusters.

For a fair likelihood-ratio comparison, the models must be nested, fitted to the same response observations, and have the same random-effects structure when the target is a fixed effect. The likelihoods must be calculated on a comparable basis; fixed-effect likelihood comparisons are ordinarily made using maximum likelihood. The Wald calculation must use the coefficient estimate and covariance matrix from the corresponding full model. If these requirements are not met, different p-values may reflect different analyses rather than different test behavior.

Why Agreement Is Expected Locally

The connection between the tests follows from a local approximation. Let \(\boldsymbol{\theta}\) denote the free parameter vector, and let \(\hat{\boldsymbol{\theta}}\) maximize the log likelihood. Near that maximum, a second-order Taylor expansion gives

\ell(\boldsymbol{\theta}) \approx \ell(\hat{\boldsymbol{\theta}}) -\frac{1}{2}(\boldsymbol{\theta}-\hat{\boldsymbol{\theta}})^\mathsf{T} \boldsymbol{I}(\hat{\boldsymbol{\theta}}) (\boldsymbol{\theta}-\hat{\boldsymbol{\theta}}),

where \(\boldsymbol{I}(\hat{\boldsymbol{\theta}})\) is the observed information matrix, the negative curvature of the log likelihood at the maximum. The Wald statistic measures the squared distance to the null using the estimated covariance matrix, which is approximately the inverse information matrix. The likelihood-ratio statistic measures twice the log-likelihood drop from the maximum to the best point satisfying the null. If the quadratic approximation is accurate across that distance, these are two expressions for approximately the same quantity.

This argument also explains the main source of disagreement: the quadratic approximation is local. A Wald test uses curvature at the maximum and projects toward the null. A likelihood-ratio test evaluates the likelihood at the constrained fit, which may be far enough away that the curvature changes. If the likelihood profile is asymmetric, the actual log-likelihood drop can differ from the quadratic prediction. In small samples, higher-order terms and uncertain estimates of curvature can matter.

Worked Example: A Wald Result Just Above the Significance Cutoff

Consider a hypothetical binary-outcome GLMM with a treatment fixed effect \(\beta\), measured on the log-odds scale. Suppose the full-model fit gives \(\hat{\beta}=0.80\), \(SE(\hat{\beta})=0.42\), and the null model fixes \(\beta=0\). The two models use the same observations and random-effects structure. Their maximized log likelihoods are \(-51.60\) for the full model and \(-53.70\) for the null model. There are no convergence warnings, and the coefficient estimate and covariance are finite.

Wald test. The estimated standard error gives

W=\left(\frac{0.80-0}{0.42}\right)^2 =1.9048^2 =3.628.

With one restriction, compare this with a \(\chi^2_1\) reference distribution: \(p\approx 0.0568\). At \(\alpha=0.05\), this Wald test does not reject \(H_0\).

Likelihood-ratio test. The log-likelihood difference is \(-51.60-(-53.70)=2.10\), so

$$ G^2=2(2.10)=4.20. $$

Against \(\chi^2_1\), \(p\approx 0.0404\); this test rejects \(H_0\) at the same significance level. The difference is plausible if the likelihood is not well approximated by a symmetric quadratic between \(\hat{\beta}=0.80\) and the null value. These summaries do not show which approximation is better; they show why inspecting the profile and fit is useful.

Coefficient Scale Can Change a Wald Test

A Wald test depends on the parameterization because it uses a local standard-error approximation on the chosen scale. A linear rescaling, such as changing a coefficient from one unit to another, leaves the scalar Wald statistic unchanged when the estimate and standard error are both rescaled consistently. A nonlinear transformation need not preserve it. In a logistic model, for example, a log-odds coefficient \(\beta\) can be reported as an odds ratio \(\exp(\beta)\). The null value \(\beta=0\) corresponds to an odds ratio of 1, but a Wald test computed directly on the odds-ratio scale can differ from one computed on the log-odds scale.

The likelihood-ratio statistic is invariant to a one-to-one reparameterization: expressing the same fitted likelihood in terms of \(\beta\) or \(\exp(\beta)\) does not change the maximized log likelihoods or their difference. This makes it a useful reference when a Wald conclusion seems sensitive to the reported scale. It does not make the likelihood-ratio test immune to small-sample problems.

Worked Example: Testing on the Log-Odds and Odds-Ratio Scales

Suppose a hypothetical GLMM for a binary outcome estimates a treatment coefficient \(\hat{\beta}=0.55\) with \(SE(\hat{\beta})=0.30\). The coefficient is in log-odds units, and the null is \(\beta=0\), equivalently an odds ratio of 1. Assume the models are nested and the maximized log likelihoods are \(-42.10\) and \(-44.15\) for the full and null fits, respectively.

On the log-odds scale, the Wald statistic is

W_\beta=\left(\frac{0.55}{0.30}\right)^2 =1.833^2 =3.361,

giving \(p\approx 0.0667\) from \(\chi^2_1\). This test does not reject at \(\alpha=0.05\).

The estimated odds ratio is \(\exp(0.55)=1.733\). Using the delta method, its approximate standard error is \(\exp(0.55)(0.30)=0.520\). A Wald calculation directly on the odds-ratio scale gives

W_{\mathrm{OR}}= \left(\frac{1.733-1}{0.520}\right)^2 =\left(\frac{0.733}{0.520}\right)^2 \approx 1.99,

with \(p\approx 0.158\). Finally, the likelihood-ratio statistic is \(2\{-42.10-(-44.15)\}=4.10\), with \(p\approx 0.0429\). This example illustrates that the transformed-scale Wald approximation can differ even when the null hypothesis describes the same effect. It does not justify choosing whichever scale produces the smaller p-value. State the parameterization, use a coherent prespecified test, and report the odds-ratio estimate with its scale and uncertainty.

Limited Cluster Information and Diagnostics

The chi-square reference distributions describe large-sample behavior, not a guarantee for every fitted GLMM. With few independent clusters, fixed-effect estimates and their standard errors may be unstable, and the likelihood surface can be irregular. A data set may contain hundreds of individual observations but only a small number of independent cluster contributions to the likelihood. This is particularly consequential when a predictor varies mainly between clusters or when outcomes are sparse in some clusters.

When Wald and likelihood-ratio results disagree, use a diagnostic sequence rather than treating the discrepancy as a vote:

Verify the comparison.
Confirm the null restriction, the degrees of freedom, the analysis observations, the fixed and random effects, and the likelihood basis. Check that the models are nested and that the intended fixed-effect term is the only difference.

Check numerical fitting.
Inspect convergence messages, gradients, Hessian or covariance warnings, and the stability of estimates under sensible starting values. A poorly optimized likelihood or nearly singular curvature estimate can make either statistic unreliable.

Inspect the likelihood shape.
Compare the coefficient estimate and standard error with the profile likelihood around the null. Strong asymmetry, a long tail, or a flat region indicates that a local quadratic Wald approximation may be inadequate.

Assess independent information.
Count the clusters and examine how the predictor and outcomes are distributed across them. A large total observation count cannot compensate automatically for very few independent clusters.

Worked Example: Many Observations but Only Six Clusters

Suppose a hypothetical study has 120 binary observations from 6 clinics, with 20 observations per clinic. A treatment fixed effect is estimated as \(\hat{\beta}=0.60\) on the log-odds scale with \(SE(\hat{\beta})=0.34\). The clinics contribute to both treatment groups, and the same observations and random-intercept structure are used in the nested fits. Assume the full and null log likelihoods are \(-76.20\) and \(-78.60\), respectively, and that fitting diagnostics show no numerical failure.

The Wald statistic is

W=\left(\frac{0.60}{0.34}\right)^2 =1.7647^2 =3.114,

so \(p\approx 0.0777\) from \(\chi^2_1\). The likelihood-ratio statistic is

G^2=2\{-76.20-(-78.60)\}=2(2.40)=4.80,

giving \(p\approx 0.0285\) from \(\chi^2_1\). The likelihood-ratio test rejects at \(\alpha=0.05\), whereas the Wald test does not. However, there are only six independent clinics. The nominal chi-square p-values should therefore be treated cautiously: neither calculation removes the uncertainty caused by limited independent cluster information. The appropriate conclusion is not that the treatment effect is definitively present, but that these asymptotic tests give differing evidence and their calibration may be weak in this design.

Choosing a Response, Not a Winner

A likelihood-ratio test often handles asymmetry better than a Wald test because it uses the likelihood at the constrained fit rather than only local curvature at the maximum. But its usual chi-square calibration is also asymptotic, and a small number of clusters can undermine it as well. Conversely, a Wald test is convenient for a single coefficient or a linear contrast, but can be particularly sensitive to an imprecise standard error, an asymmetric profile, or a nonlinear parameter scale.

If the discrepancy matters for the scientific conclusion, report the estimate and its uncertainty, the test statistic and reference distribution, and the modeling and fitting checks. Consider a suitable small-sample calibration, such as a parametric bootstrap under the null, when it is supported by the design and model. Such a calibration must reproduce the relevant clustering and refit the models as required; it is not a guarantee against model misspecification. Do not switch tests after seeing which one gives the preferred conclusion.

Common mistake: “The likelihood-ratio p-value is smaller, so the Wald test must be wrong.”
Correction: Both tests rely on approximations. Check model comparability, optimization, likelihood shape, parameter scale, and the number of independent clusters before interpreting either result.
Important distinction: The discussion here concerns regular fixed-effect restrictions. Tests of random-effect variances can involve boundary parameters and different reference distributions; the ordinary fixed-effect chi-square argument should not be transferred automatically.

Key takeaway: Wald and likelihood-ratio tests agree most closely when the likelihood is regular and nearly quadratic, the parameterization is well behaved, and independent information is ample. Disagreement in a small-sample GLMM is a diagnostic signal—not a reason to select the more favorable p-value.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A one-coefficient Wald test gives \(W=2.89\). What is its approximate reference distribution, and does it reject at \(\alpha=0.05\)?
For nested models, \(\ell_{\mathrm{full}}=-30.8\) and \(\ell_{\mathrm{null}}=-33.0\). Calculate the likelihood-ratio statistic for one restriction.
Why can a Wald test on an odds-ratio scale differ from a Wald test on the corresponding log-odds scale?
A GLMM has 500 observations but only 5 independent clusters. Why is the observation count alone insufficient reassurance about chi-square calibration?

Solutions. (1) Under the usual regularity conditions, \(W\) is compared with \(\chi^2_1\). Its p-value is about \(0.089\), so it does not reject at \(0.05\). (2) \(G^2=2\{-30.8-(-33.0)\}=2(2.2)=4.40\). With one restriction, the \(\chi^2_1\) p-value is about \(0.036\). (3) The odds ratio is a nonlinear transformation of the log-odds coefficient, and the Wald calculation uses a local standard-error approximation that need not be preserved by nonlinear transformations. (4) Observations within a cluster are linked through the shared random effect; the number of independent clusters is only 5, so asymptotic approximations based on many independent cluster contributions may be poor despite the large total observation count.