Wald Confidence Intervals for NLME Parameters

Why Wald Intervals Are Useful

An NLME fit reports parameter estimates, but an estimate alone does not show how precisely the data determine a parameter. A confidence interval summarizes that uncertainty on the parameter’s estimation scale. The Wald interval is a direct calculation: take a maximum-likelihood estimate and extend a chosen number of estimated standard errors in each direction.

The calculation is simple, but its interpretation depends on an approximation. In a regular model with sufficient information, the maximum-likelihood estimator is approximately normal near the true parameter value. A Wald interval uses that local normal approximation and an estimated covariance matrix. Its coverage is therefore approximate, not an exact finite-sample guarantee.

Definition: For a scalar parameter $\theta_j$, a two-sided $100(1-\alpha)\%$ Wald confidence interval is $$ \hat{\theta}_j \ \pm\ z_{1-\alpha/2}\,\widehat{SE}(\hat{\theta}_j), $$ where $\hat{\theta}_j$ is its maximum-likelihood estimate, $\widehat{SE}(\hat{\theta}_j)$ is its estimated standard error, and $z_{1-\alpha/2}$ is the $1-\alpha/2$ quantile of the standard normal distribution. The interval is pointwise: it targets the coverage of this one parameter, not simultaneous coverage of all parameters.

From the Estimated Covariance Matrix to an Interval

Let $\boldsymbol{\theta}$ denote the vector of parameters estimated by maximizing the NLME marginal likelihood, and let $\hat{\boldsymbol{\theta}}$ be its maximum-likelihood estimate. Write $\ell(\boldsymbol{\theta})$ for the marginal log likelihood. The observed information at the estimate is the negative Hessian,

J(\hat{\boldsymbol{\theta}}) = -\left.\frac{\partial^2 \ell(\boldsymbol{\theta})} {\partial \boldsymbol{\theta}\,\partial \boldsymbol{\theta}^{\mathsf T}}\right|_{\boldsymbol{\theta}=\hat{\boldsymbol{\theta}}}.

Under standard likelihood regularity conditions, the inverse observed information estimates the covariance matrix of the estimator:

\widehat{\operatorname{Cov}}(\hat{\boldsymbol{\theta}}) = \hat V = J(\hat{\boldsymbol{\theta}})^{-1}.

For component $j$, the estimated variance is the $j$th diagonal entry of $\hat V$, and the standard error is its square root. Off-diagonal entries describe estimated covariation between parameter estimates; they are not themselves standard errors.

\widehat{SE}(\hat{\theta}_j)=\sqrt{\hat V_{jj}}.

This covariance estimate is the curvature-based approximation discussed in the earlier material on turning the Hessian into a parameter covariance matrix. It is not an exact identity for finite data. The calculation presumes a suitably identified, interior maximum, a differentiable likelihood with informative curvature, and a sufficiently accurate local quadratic approximation. For NLME data, repeated records within one subject are not automatically independent units: the likelihood and its uncertainty calculation must respect the subject-level dependence in the model.

Why the Formula Has This Form

The justification is standard likelihood asymptotics applied to the marginal likelihood. Under regularity conditions and increasing information, the estimator is approximately multivariate normal with mean near the true parameter and covariance estimated by $\hat V$. For one component, this gives the approximate standardized quantity

\frac{\hat{\theta}_j-\theta_j}{\widehat{SE}(\hat{\theta}_j)} \ \approx\ N(0,1).

A standard normal variable lies between $-z_{1-\alpha/2}$ and $z_{1-\alpha/2}$ with probability $1-\alpha$. Applying those limits to the standardized estimator and rearranging yields the Wald interval:

P\!\left( -z_{1-\alpha/2} \leq \frac{\hat{\theta}_j-\theta_j}{\widehat{SE}(\hat{\theta}_j)} \leq z_{1-\alpha/2} \right) \approx 1-\alpha $$ $$ \Longrightarrow\quad P\!\left( \hat{\theta}_j-z_{1-\alpha/2}\widehat{SE}(\hat{\theta}_j) \leq \theta_j \leq \hat{\theta}_j+z_{1-\alpha/2}\widehat{SE}(\hat{\theta}_j) \right) \approx 1-\alpha.

The probability statement describes the procedure over repeated data sets generated under the model and its assumptions. It does not mean that, after the data are observed, there is a $1-\alpha$ probability that the fixed parameter lies in this particular interval. The approximation is most credible when the likelihood is locally close to quadratic and the estimator is close to normally distributed.

Worked Example: A 95% Interval for Clearance

Suppose a hypothetical NLME analysis estimates typical clearance as $\widehat{CL}=5.20$ L/h, with estimated standard error $0.60$ L/h. Assume the fit has converged to an interior maximum, the relevant curvature estimate is positive, and the local normal approximation is considered reasonable. Calculate a two-sided 95% Wald interval.

For a 95% interval, $\alpha=0.05$ and $z_{0.975}=1.96$. The margin of error is $$ 1.96(0.60\ \mathrm{L/h})=1.176\ \mathrm{L/h}. $$

Thus, $$ 5.20\ \mathrm{L/h}\ \pm\ 1.176\ \mathrm{L/h} = [4.024,\ 6.376]\ \mathrm{L/h}. $$

Rounded to three significant figures, the approximate 95% Wald interval is $[4.02,\ 6.38]$ L/h. Under the model and the local normal approximation, this interval procedure would cover the true typical clearance in approximately 95% of repeated data sets. The interval is symmetric around the estimate on the clearance scale.

Worked Intervals for Multiple Parameters

An NLME model usually estimates multiple parameters together. The diagonal of their estimated covariance matrix gives the marginal standard errors needed for individual Wald intervals. The off-diagonal elements can matter for uncertainty in combinations of parameters, but they do not change the formula for an individual component’s interval.

Worked Example: Using Diagonal Entries of a Covariance Matrix

For a hypothetical fit, suppose the estimates are typical clearance $\hat{\theta}_1=5.20$ L/h and a second fixed effect $\hat{\theta}_2=0.80$ (dimensionless). The estimated covariance matrix, with units corresponding to these parameters, is $$ \hat V= \begin{pmatrix} 0.36 & 0.12\\ 0.12 & 0.25 \end{pmatrix}. $$ Assume the maximum is interior and the estimated covariance matrix is positive definite. Find separate 95% Wald intervals.

The standard errors are the square roots of the diagonal entries: $$ \widehat{SE}(\hat{\theta}_1)=\sqrt{0.36}=0.60\ \mathrm{L/h}, \qquad \widehat{SE}(\hat{\theta}_2)=\sqrt{0.25}=0.50. $$ Using $z_{0.975}=1.96$, the interval for clearance is $$ 5.20\pm 1.96(0.60) = 5.20\pm1.176 = [4.024,\ 6.376]\ \mathrm{L/h}. $$ The interval for the second parameter is $$ 0.80\pm1.96(0.50) = 0.80\pm0.98 = [-0.18,\ 1.78]. $$

These are two marginal, pointwise intervals; they do not together form a 95% confidence region for both parameters. The covariance $0.12$ indicates that the estimates covary in the fitted approximation, but it is not substituted for either standard error. The assumptions about a well-behaved maximum and local normality apply to both interval calculations.

Worked Example: Interpreting an Interval That Includes Zero

Suppose a hypothetical model estimates a dimensionless fixed effect $\hat{\beta}=-0.12$, representing the difference in a model parameter between two groups on the specified estimation scale. Its estimated standard error is $0.09$. Assume the estimate is interior, the likelihood curvature is informative, and the local normal approximation is adequate. Calculate and interpret a 95% Wald interval.

The margin of error is $$ 1.96(0.09)=0.1764. $$ Therefore, $$ -0.12\pm0.1764=[-0.2964,\ 0.0564]. $$

The approximate 95% interval is $[-0.296,\ 0.0564]$ on the dimensionless scale. Zero lies inside it, so the interval includes values corresponding to no group difference as well as values on either side of zero. This does not prove that the groups are identical or that the effect is zero; it indicates that this interval procedure does not exclude zero at the stated level. Interpretation should remain on the parameter scale specified by the model.

Checks, Alternatives, and Limitations

Before reporting a Wald interval, check that its ingredients are numerically credible. The estimate should come from a fit that has converged to a plausible maximum, and the observed information should support a finite, positive variance estimate for the component. A non-positive diagonal variance, a non-positive-definite information matrix, or a very large standard error relative to the estimate can indicate weak identification, problematic curvature, or an optimization issue. These are reasons to investigate the fit, not reasons to report an interval mechanically.

The local approximation deserves particular caution when the likelihood is strongly asymmetric, when an estimate is near a parameter boundary, or when the data contain little information about a parameter. Since a Wald interval extends equally far above and below the estimate, it can also produce endpoints that are implausible for the parameter’s allowed range. Such behavior signals that the interval’s approximation may be unsuitable; the interval itself does not enforce parameter constraints.

Wald intervals are attractive because they are quick to calculate and use the estimate and standard error already obtained from the fit. A profile-likelihood interval instead uses changes in the likelihood as a parameter is varied, and bootstrap intervals use the distribution of estimates across resampled or simulated data sets. Those alternatives can represent uncertainty differently from a local symmetric approximation, but require additional computation and their own assumptions and diagnostics. A Wald interval is therefore a useful first summary, not a universal replacement for other uncertainty methods.

Check the scale. The interval is symmetric on the scale where $\hat{\theta}_j$ and its standard error are reported.
Check the curvature. A covariance estimate should be finite and consistent with a well-defined maximum.
Check the scope. A 95% interval for one parameter is not a 95% joint region for every parameter in the model.
Check the interpretation. The confidence level concerns repeated-sampling coverage under the assumed model, not the probability that a fixed parameter is in the observed interval.

Key takeaway: A two-sided Wald interval combines an MLE with its covariance-based standard error and a standard-normal critical value. Its simple form is justified by an approximate local normal distribution, so its credibility depends on regularity, informative curvature, and a suitably accurate local approximation.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A hypothetical parameter estimate is $2.40$ with standard error $0.30$. Calculate its approximate two-sided 95% Wald interval.
For an estimated covariance matrix, the third diagonal entry is $0.49$. What is the standard error of the third parameter estimate?
In repeated sampling, what does approximate 95% coverage mean, and what does it not mean about one observed interval?
Why does an off-diagonal covariance entry not serve as the standard error for either of its two parameters?

Solutions. (1) The margin is $1.96(0.30)=0.588$, so the interval is $2.40\pm0.588=[1.812,\ 2.988]$, or approximately $[1.81,\ 2.99]$. (2) The standard error is the square root of the variance: $\sqrt{0.49}=0.70$. (3) Under the model and approximation, about 95% of intervals formed by this procedure across repeated data sets would cover the fixed parameter. It does not assign a 95% probability that the parameter lies in the particular interval already calculated. (4) An off-diagonal entry estimates covariance between two parameter estimates; the standard error of one component is the square root of its own diagonal variance entry.