Why Wald Tests Fail for Variance Components

Why a Familiar Test Can Mislead

A Wald test is attractive because its calculation is simple: divide an estimate by its standard error and compare the result with a normal reference distribution. For a fixed-effect coefficient well inside its parameter space, that approximation can be useful when the model is adequately specified and the sample is large enough. A random-effect variance is different. It cannot be negative, and the null hypothesis that it is zero lies exactly at the edge of the allowed parameter space.

That edge matters. When the true variance is zero, sampling variation cannot produce a negative fitted variance; instead, a constrained fitting procedure may return exactly zero. Near zero, estimates can therefore be asymmetric, concentrated at the boundary, and far from the bell-shaped distribution assumed by an ordinary Wald test. The standard error used in the calculation can also become unstable. These are structural problems, not just small-sample inconveniences.

Consider a GLMM with conditional mean specified through a link function and random effects $b_i$, with $b_i\sim N(\boldsymbol 0,\boldsymbol G)$. A component variance $\tau$ in $\boldsymbol G$ must satisfy $\tau\geq 0$. The question here is not how to fit that model, but why the usual Wald approximation for testing $H_0:\tau=0$ can be poorly calibrated.

Definition: A Wald test for a scalar parameter $\tau$ uses the estimate $\hat{\tau}$ and its estimated standard error $SE(\hat{\tau})$. For testing a specified value $\tau_0$, the statistic is $$ Z_W=\frac{\hat{\tau}-\tau_0}{SE(\hat{\tau})}. $$ The usual normal approximation treats $Z_W$ as approximately $N(0,1)$ under $H_0$. That approximation relies on regularity conditions, including an interior true parameter value, a sufficiently regular likelihood, and a stable, nonzero information estimate.

The Boundary Changes the Sampling Distribution

For an interior value $\tau>0$, a sufficiently large-sample estimator may be approximately normal around the true value. At $\tau=0$, however, the parameter is not interior: values below zero are prohibited. The unconstrained estimate may fluctuate on both sides of zero, but the constrained estimate cannot. As a result, the ordinary symmetric normal approximation does not describe the constrained estimate in general.

A simple quadratic-likelihood calculation shows the mechanism. It is an idealized illustration, not a claim that every fitted GLMM has this exact distribution. Suppose an unconstrained estimate $Y$ has $Y/s=Z$, where $Z\sim N(0,1)$, and the log likelihood near its maximum is proportional to $-(\tau-Y)^2/(2s^2)$. Maximizing this expression subject to $\tau\geq0$ gives the constrained estimate:

\hat{\tau}=\max(0,Y), \qquad \frac{\hat{\tau}}{s}=\max(0,Z).

When $Z\leq0$, the constrained estimate is zero; when $Z>0$, it equals the positive unconstrained estimate. Under this idealized boundary model, $P(\hat{\tau}=0)=P(Z\leq0)=0.5$. The positive estimates follow only the positive side of a normal curve, not a symmetric normal distribution centered at zero. This demonstrates why ordinary normal-theory reasoning needs special care at a boundary.

There is an important qualification: in this especially simple calculation, a one-sided upper-tail cutoff can happen to give the nominal upper-tail probability, because $\hat{\tau}/s>c$ is equivalent to $Z>c$ for $c>0$. Real GLMM Wald calculations generally do not have this exact known-$s$, quadratic-likelihood structure. They use estimated standard errors, potentially nonlinear likelihoods, and nuisance parameters. The idealized calculation explains the boundary effect; it does not certify the calibration of a routine Wald test.

Key takeaway: A Wald test is not automatically valid just because software reports an estimate and standard error. At a variance boundary, the normal approximation and the estimated information must both be scrutinized; neither is guaranteed to behave as it does for an interior fixed-effect coefficient.

Worked Calculations and Warning Signs

Worked Example: A Naive Wald Test for a Random-Intercept Variance

Suppose a hypothetical binary-outcome GLMM reports a random-intercept variance estimate $\hat{\tau}=0.0400$ with $SE(\hat{\tau})=0.0300$, on the log-odds-squared scale. Consider $H_0:\tau=0$ against $H_a:\tau>0$. First calculate the ordinary Wald statistic:

Z_W=\frac{0.0400-0}{0.0300}=1.3333.

If one nevertheless uses the one-sided standard-normal reference, the nominal p-value is $P(Z\geq4/3)\approx0.09121$, where $Z\sim N(0,1)$. This exceeds a nominal $\alpha=0.05$, so this calculation does not reject $H_0$. The p-value is the probability, under the reference model and null, of a statistic at least as large as the observed one; it is not the probability that the variance is zero.

The number $0.09121$ is only the p-value under the ordinary normal approximation. Because the null places $\tau$ on the boundary, that approximation needs justification before the result is treated as a calibrated test. The calculation is easy; its reference distribution is the concern.

A normal-based 95% interval makes the constraint problem visible:

0.0400\pm1.96(0.0300) =[-0.0188,\ 0.0988].

The lower limit is negative, although a variance cannot be negative. This does not prove that the Wald p-value is wrong by a particular amount. It does show that the symmetric interval construction does not respect the parameter space and is a warning against treating the estimate as an ordinary interior parameter.

Worked Example: Wald Results Can Depend on the Parameterization

Suppose another hypothetical GLMM gives a variance estimate $\hat{\tau}=0.0100$ with $SE(\hat{\tau})=0.0200$. One might report the random-effect standard deviation $q=\sqrt{\tau}$ instead. The variance-scale Wald statistic is

Z_{W,\tau}=\frac{0.0100}{0.0200}=0.500.

Its one-sided normal-reference p-value is $P(Z\geq0.500)\approx0.3085$. Now apply the delta method away from zero to obtain an approximate standard error for $q$. Since $dq/d\tau=1/(2\sqrt{\tau})$, the plug-in standard error is

SE(\hat q)\approx \frac{SE(\hat{\tau})}{2\sqrt{\hat{\tau}}} =\frac{0.0200}{2(0.1000)} =0.1000.

The estimated standard deviation is $\hat q=\sqrt{0.0100}=0.1000$, so its Wald statistic against zero is $0.1000/0.1000=1.000$, with one-sided normal-reference p-value $P(Z\geq1.000)\approx0.1587$. These two Wald p-values differ, even though $q=\sqrt{\tau}$ is a one-to-one transformation for positive values.

This disagreement is a sign of trouble, not evidence that one calculation must be correct. Wald approximations are not generally invariant to nonlinear reparameterization. More fundamentally, the delta-method derivative becomes unbounded as $\tau$ approaches zero, exactly where the null lies. For reference, the variance-scale 95% Wald interval is $0.0100\pm1.96(0.0200)=[-0.0292,0.0492]$, again including impossible negative variances. Truncating that interval at zero would enforce the parameter constraint but would not, by itself, establish correct coverage.

Worked Example: A Pile at Zero and a Long Positive Tail

Imagine fitting the same hypothetical GLMM to ten independently generated datasets under a true random-effect variance of zero. Suppose the fitted variances, in the model's variance units, are $0,0,0,0.002,0.004,0.009,0.015,0.030,0.052,0.090$. These are illustrative repeated-fit results, not observations from a real study.

Three of the ten estimates are exactly zero, so the observed fraction at the boundary is $3/10=0.300$. The mean estimate is

\bar{\tau} =\frac{0+0+0+0.002+0.004+0.009+0.015+0.030+0.052+0.090}{10} =\frac{0.202}{10} =0.0202.

The median is the average of the fifth and sixth ordered values: $(0.004+0.009)/2=0.0065$. The mean is much larger than the median because a few positive estimates extend far above the zeros and small values. This pattern—boundary estimates combined with a long positive tail—is unlike a symmetric bell-shaped sampling distribution.

Ten fits are not enough to calibrate a test or estimate a tail probability reliably. The exercise instead identifies what to inspect in a larger simulation or fitting diagnostic: how often estimates hit zero, how asymmetric the positive estimates are, and whether reported standard errors track their sampling variability. A single estimate and standard error cannot reveal all of this distributional behavior.

What Fails, and What to Check

The standard Wald justification depends on an approximately normal estimator and a reliable estimate of its variance. At a variance boundary, both elements can fail. The constrained estimator may have a point mass at zero and a skewed positive tail. The curvature of the likelihood can be weak near zero, making the estimated standard error very large, unstable, or unavailable. In some fits the software reports a boundary estimate but a Hessian-based standard error that is not meaningful; in others, a small change in data or parameterization can materially change the Wald statistic.

These issues are especially concerning when there are few independent clusters, when the variance estimate is close to zero, or when a random-effects covariance structure is nearly singular. A numerically converged model is not proof that the Wald approximation is adequate. Convergence checks address optimization; they do not establish the reference distribution for a test.

Useful warning signs include:

A variance estimate equal or very close to zero, or repeated fits that frequently return zero.
A Wald interval with a negative lower endpoint or a normal approximation that assigns substantial probability to impossible values.
A standard error that changes sharply with small modeling or numerical changes, or that is not reported because the information matrix is singular.
Substantially different Wald conclusions after transforming from variance to standard deviation.
Strong asymmetry or a long right tail in repeated-fit estimates under conditions similar to the analysis.

These signs do not all have the same meaning. For example, a negative lower Wald limit is a diagnostic of the symmetric construction, not a direct measure of the p-value's error. The appropriate response is to question the approximation and use an inferential approach whose assumptions match the boundary problem, rather than to repair the result by simply clipping a negative estimate or interval endpoint.

How Wald Inference Compares with Alternatives

A Wald test is most defensible when the parameter is well inside its allowed range, the likelihood is locally close to quadratic, and the standard error is stable. Those conditions are much less plausible for testing a variance equal to zero. A likelihood-ratio test is based on the change in maximized likelihood between models; at a boundary, its reference law may need to be a boundary-aware mixture rather than the ordinary chi-square law. The earlier material on likelihood-ratio tests for variance components develops that issue.

Other approaches include profile likelihood and parametric bootstrap calibration. Each has its own assumptions and computational or finite-sample limitations; neither is automatically correct simply because it avoids the elementary Wald ratio. The choice depends on the variance structure, nuisance parameters, number of clusters, and the reliability of the fitted model. The relevant lesson here is to distinguish a convenient statistic from a justified calibration.

In particular, do not interpret a Wald p-value near or above a threshold as strong evidence that the variance is absent, nor a small Wald p-value as automatically conclusive evidence of heterogeneity. A variance estimate of zero may reflect the constraint and the information in the data; a nonzero estimate may be highly uncertain. Report the estimate and its scale, describe the boundary issue, and base inferential claims on a method appropriate to the design and model.

Common pitfall: “The software reported $\hat{\tau}/SE(\hat{\tau})=1.33$, so the standard-normal p-value is valid.” The division is correct, but the conclusion skips the key assumption: that the statistic has the stated normal reference distribution. At $\tau=0$, the parameter is on a boundary, estimates can pile up at zero, and the standard error can be unstable. Calculate the Wald quantity if useful, but do not treat its normal p-value as calibrated without a boundary-specific justification.

A Practical Assessment

Locate the null in the parameter space.
For a variance-component test of zero, recognize that zero is the boundary, not an ordinary interior value.

Inspect the estimate and its uncertainty.
Check for a boundary estimate, a negative symmetric interval endpoint, unstable standard errors, and sensitivity to parameterization.

Separate calculation from calibration.
The estimate divided by its standard error is a computable Wald statistic; the normal reference distribution still requires justification.

Choose a boundary-aware inference strategy.
Compare available methods in light of the model, number of clusters, nuisance parameters, and computational constraints, and report the limitations.

Check Your Understanding

Answer each question, then compare with the concise solutions below.

A fitted variance is $0.0600$ with standard error $0.0200$. Calculate the Wald statistic for testing zero and its one-sided standard-normal p-value. What does the calculation not establish?
Why can a fitted variance be exactly zero even when the data are random?
A variance-scale Wald interval extends below zero. Is the negative endpoint a possible variance, and does replacing it with zero guarantee a valid interval?
Why can Wald tests for variance and standard deviation give different results for the same random-effect term?
Name two features of repeated fits that would make an ordinary normal approximation concerning.

Solutions. (1) $Z_W=0.0600/0.0200=3.000$, and $P(Z\geq3.000)\approx0.00135$. This is the p-value under the standard-normal reference; the calculation alone does not establish that this reference is calibrated at a variance boundary. (2) The parameter is constrained to be nonnegative, so an unconstrained estimate below zero can be set to the boundary by constrained fitting. (3) A negative variance is impossible. Truncating the endpoint at zero enforces the parameter space but does not prove the resulting interval has its advertised coverage. (4) A nonlinear transformation changes the local scale and the delta-method standard error; Wald approximations are not generally invariant, and the transformation is especially problematic near zero. (5) Examples include a frequent pile of estimates at zero, a strongly right-skewed distribution, unstable standard errors, or marked sensitivity to parameterization.