When Wald Intervals Fail in NLME Models

Why a Wald Interval Can Look Precise but Be Unreliable

A Wald confidence interval is built from an estimate and its standard error. Its compact form can make it look definitive, but the interval is only as trustworthy as the local normal approximation used to construct it. In a nonlinear mixed-effects (NLME) model, that approximation can be poor when the likelihood is asymmetric, a parameter is weakly identified, an estimate is near a boundary, or parameters are strongly correlated.

The issue is not simply that an interval is wide or narrow. A Wald interval can be narrow while the likelihood has a long, flat ridge in a different direction; it can extend into impossible values; or it can place its endpoints in regions with very different likelihood support. The goal is therefore to diagnose whether the approximation fits the likelihood and the parameter space before interpreting the interval.

Definition: For a scalar parameter \(\theta_j\), a two-sided Wald interval is \(\hat{\theta}_j \pm z_{1-\alpha/2}SE(\hat{\theta}_j)\), where \(\hat{\theta}_j\) is the estimate, \(SE(\hat{\theta}_j)\) is its estimated standard error, and \(z_{1-\alpha/2}\) is the corresponding standard-normal quantile. Its usual large-sample interpretation relies on the estimator being approximately normal around the true parameter, with the standard error estimating its sampling variability.

In an NLME model, likelihood-based standard errors are commonly obtained from the inverse observed information at the fitted estimates. The relevant likelihood is the marginal likelihood: individual random effects are integrated out rather than treated as known. The likelihood is a function of the population parameters, including fixed effects, random-effects parameters, and residual-error parameters. A Wald interval is most defensible when the maximum is well determined, the parameter is away from boundaries, the likelihood is smooth and nearly quadratic over the interval, and the fitted information matrix is reliable.

What the Local Quadratic Approximation Requires

Let \(\boldsymbol{\theta}\) be the vector of model parameters and let \(\ell(\boldsymbol{\theta})\) be the marginal log likelihood. Near an interior maximum \(\hat{\boldsymbol{\theta}}\), a second-order Taylor approximation is

\ell(\boldsymbol{\theta}) \approx \ell(\hat{\boldsymbol{\theta}}) - \frac{1}{2}(\boldsymbol{\theta}-\hat{\boldsymbol{\theta}})^\mathsf{T} J(\hat{\boldsymbol{\theta}}) (\boldsymbol{\theta}-\hat{\boldsymbol{\theta}}),

where \(J(\hat{\boldsymbol{\theta}})\) is the observed information, the negative Hessian matrix of the log likelihood at the estimate. If it is positive definite and the approximation is adequate, the estimated covariance matrix is \(J(\hat{\boldsymbol{\theta}})^{-1}\). A diagonal element of this inverse gives the estimated variance of one parameter, accounting for its joint estimation with the others.

Key distinction: In a one-parameter model, moving \(z\) standard errors from the estimate under the quadratic approximation reduces the log likelihood by approximately \(z^2/2\). The same statement applies to a profile-likelihood change when nuisance parameters are re-optimized and the profile is locally quadratic. It does not generally apply to a change in one coordinate while all other parameters are held fixed in a multivariate likelihood.

The distinction matters because the curvature of a multivariate likelihood depends on direction. Marginal standard errors come from the inverse information matrix; the curvature along a fixed-coordinate direction comes from the information matrix itself. Strong correlation can make those curvatures very different. The following derivation makes the distinction explicit.

Why Correlation Changes the Curvature

Partition the parameters into a scalar of interest \(\psi\) and nuisance parameters \(\boldsymbol{\lambda}\). Write the quadratic information matrix in blocks:

J = \begin{pmatrix} J_{\psi\psi} & J_{\psi\lambda}\\ J_{\lambda\psi} & J_{\lambda\lambda} \end{pmatrix}.

If nuisance parameters are held fixed, a change \(\delta\) in \(\psi\) has approximate log-likelihood drop \(\tfrac12 J_{\psi\psi}\delta^2\). If instead they are re-optimized, maximizing the quadratic approximation over their changes gives the profile curvature

J_{\mathrm{profile}} = J_{\psi\psi} - J_{\psi\lambda}J_{\lambda\lambda}^{-1}J_{\lambda\psi}.

This follows by completing the square, or by setting the derivative with respect to the nuisance change to zero and substituting its maximizing value. The scalar marginal variance is \(1/J_{\mathrm{profile}}\), not generally \(1/J_{\psi\psi}\). Thus a marginal standard error describes uncertainty after allowing joint parameter variation; it does not describe the curvature of a cut through the likelihood with every other parameter frozen.

Worked Example: A Fixed-Coordinate Change in a Correlated Model

Suppose two estimated log parameters have covariance matrix

\widehat{\mathrm{Cov}}(\hat{\boldsymbol{\eta}})= \begin{pmatrix} 0.0400 & 0.0392\\ 0.0392 & 0.0400 \end{pmatrix}.

Each marginal standard error is \(\sqrt{0.0400}=0.200\), and the estimated correlation is \(0.0392/(0.200\cdot0.200)=0.980\). Consider a change of \(1.96\) marginal standard errors in the first parameter, \(\delta=1.96(0.200)=0.392\), while holding the second parameter fixed.

The determinant of the covariance matrix is \(0.0400^2-0.0392^2=0.00006336\). Its inverse therefore has first diagonal entry \(0.0400/0.00006336\approx631.31\). The quadratic log-likelihood drop for this fixed-coordinate change is

\frac{1}{2}(0.392)^2(631.31)\approx 48.50.

That is far larger than \(1.96^2/2=1.921\). There is no contradiction: the value \(1.921\) applies to the marginal, locally quadratic change with the nuisance parameter re-optimized. In this example the corresponding profile curvature is \(1/0.0400=25\), giving a profile log-likelihood drop of \(\tfrac12(0.392)^2(25)=1.921\). Strong correlation makes the fixed-coordinate cut steep even though the likelihood is much flatter along a direction where both parameters change together.

Common Reasons Wald Intervals Fail

Skewed likelihood. A symmetric interval around an estimate cannot represent unequal likelihood support on either side. This is common for positive parameters with substantial relative uncertainty.
Weak identifiability. The data may constrain a combination of parameters but not the individual values well. A nearly flat direction or ridge makes the information matrix unstable, so its inverse and resulting standard errors can be sensitive to small changes.
Parameter boundaries. Variance components cannot be negative, proportions lie between zero and one, and some rate or clearance parameters must be positive. Near a boundary, a symmetric normal approximation may assign substantial weight to impossible values and may fail because the usual interior approximation does not apply.
Strong parameter correlation. Individual marginal intervals can conceal a narrow, tilted joint likelihood region. A parameter estimate may change substantially when a correlated parameter is adjusted.
Poor local quadratic fit. Even with an interior estimate and an invertible Hessian, curvature at a single point may not describe likelihood shape across the interval. A small standard error does not prove that the approximation remains accurate that far from the maximum.

Worked Example: A Wald Interval Crossing a Boundary

Suppose a hypothetical NLME fit estimates a random-effects variance component \(\omega^2=0.0020\), with standard error \(0.0080\), in squared units of the associated parameter. The nominal 95% Wald interval is

0.0020 \pm 1.96(0.0080) =0.0020\pm0.01568 =[-0.01368,\ 0.01768].

The lower endpoint is negative, although a variance cannot be negative. This is a warning that a symmetric interval based on an interior normal approximation is not suitable here. It is not appropriate to interpret the negative endpoint as a scientifically possible variance or to conclude that the model estimates a negative variance. The estimate is close to the boundary relative to its standard error, and the likelihood may be asymmetric or constrained. A boundary-aware likelihood analysis or a suitable resampling-based interval may be more informative, while recognizing that the choice must match the model and inferential target.

Worked Example: Back-Transformation Does Not Guarantee Accuracy

Suppose a positive rate constant is estimated on the log scale as \(\hat{\eta}=-2.120\), with \(SE(\hat{\eta})=0.550\), where \(\eta=\log(\theta)\) and \(\theta\) is measured in \(\mathrm{h}^{-1}\). The estimate on the original scale is \(\hat{\theta}=\exp(-2.120)\approx0.1200\ \mathrm{h}^{-1}\). A 95% Wald interval on the log scale is

-2.120 \pm 1.96(0.550) =[-3.198,\ -1.042].

Exponentiating the endpoints gives \([\exp(-3.198),\exp(-1.042)]\approx[0.0409,\ 0.3528]\ \mathrm{h}^{-1}\). This interval is positive and asymmetric around \(0.1200\ \mathrm{h}^{-1}\), unlike a symmetric interval formed directly on the original scale. But positivity and asymmetry alone do not establish reliability. The interval still depends on a locally quadratic, approximately normal likelihood for \(\eta\). If that likelihood is skewed, weakly identified, or poorly approximated over the interval, the transformed Wald interval can also misrepresent uncertainty.

Practical Diagnostics and Alternatives

Before reporting a Wald interval, check whether it is compatible with the parameter’s allowed range, whether the estimate is close to a boundary, and whether the covariance matrix is numerically stable. Inspect correlations and the eigenvalues or condition of the information matrix: near-singularity suggests that some parameter direction is poorly determined. Also examine whether modest changes in starting values, model parameterization, or fitting tolerances materially alter the estimate or its standard error. Such sensitivity is a warning, not proof by itself, that the local approximation is failing.

Where feasible, evaluate the likelihood away from the maximum to see whether its shape is approximately quadratic and symmetric over the proposed interval. For correlated parameters, assess joint behavior rather than relying only on separate marginal intervals. These checks diagnose the approximation; they do not turn a Wald interval into a valid one if the likelihood is strongly nonquadratic or the estimate is on a boundary.

The main alternatives address uncertainty differently. Likelihood-based intervals use the likelihood’s shape away from its maximum and can represent asymmetry; they require reliable constrained optimization and appropriate treatment of boundaries. Bootstrap intervals use repeated resampled or simulated data sets to approximate the estimator’s sampling distribution; their validity depends on the resampling design and the stability of model refits. Wald intervals remain convenient and can be effective for well-identified, interior parameters with nearly quadratic likelihoods. No alternative automatically solves a structural lack of information in the data.

Approach	What it relies on	Warning sign it may be unsuitable
Wald interval	Approximate normality and local quadratic likelihood	Boundary estimate, strong skewness, unstable information, or impossible endpoints
Likelihood-based interval	Reliable likelihood evaluation over a range of parameter values	Optimization fails, constraints are mishandled, or the model is not identifiable
Bootstrap interval	Resamples or simulated data that reproduce the relevant study structure and stable refitting	Many failed fits or a resampling design that does not reflect the sampling units

Common Mistake and Key Takeaway

Common mistake: “The parameter moved by 1.96 standard errors, so the log likelihood must have dropped by about 1.92.” That reasoning is valid for a one-parameter quadratic likelihood, or for the corresponding locally quadratic profile change with nuisance parameters re-optimized. It is not generally valid when one coordinate changes while correlated parameters are held fixed. In a multivariate model, the inverse information determines marginal standard errors, whereas the information matrix determines curvature along fixed-coordinate directions.

A Wald interval is a useful summary when its assumptions are plausible, not a universal default whose validity follows from a converged fit. Check the parameter space, likelihood shape, identifiability, correlation, and numerical stability. If these checks reveal a problem, report the limitation and use an uncertainty method suited to the likelihood and study design.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A 95% Wald interval for a positive rate parameter includes negative values. What does this indicate, and what does it not indicate?
Why can a fixed-coordinate likelihood cut be steeper than the profile likelihood for the same parameter?
A parameter estimate and its standard error are stable, but the likelihood is visibly asymmetric across the proposed interval. Is the Wald interval necessarily reliable? Explain.
Name two diagnostics, other than checking whether an interval contains impossible values, that can reveal an unreliable Wald approximation.

Solutions. (1) The symmetric normal approximation is warning of a boundary or poor scale for inference; the negative endpoint is not a possible value and is not evidence that the parameter is negative. (2) A fixed-coordinate cut holds correlated nuisance parameters constant, while profiling re-optimizes them. The joint likelihood can be much flatter along a direction where parameters change together. (3) No. Stability of the estimate and standard error does not make the likelihood quadratic; asymmetry means the equal-distance endpoints need not have comparable likelihood support. (4) Examples include inspecting parameter correlations and information-matrix conditioning, checking sensitivity to starting values or fitting tolerances, and examining likelihood behavior away from the maximum.