Joint Confidence Regions for Correlated NLME Parameters

Why Joint Uncertainty Matters

In a nonlinear mixed-effects model, fixed effects often describe related features of a typical individual. For example, typical clearance and typical volume of distribution may be estimated together, and their estimates may be correlated. Reporting uncertainty for each parameter separately does not show which pairs of values are jointly compatible with the data. That distinction matters when interpreting combinations of parameters or comparing a proposed pair of values with the fitted model.

A joint confidence region addresses the pair as a vector. Its shape reflects both the uncertainty in each estimate and the estimated covariance between them. The main result in this tutorial is an approximate Wald confidence ellipse, which is useful when the estimator is approximately multivariate normal and the covariance estimate is reliable.

Definition: Let $\theta=(\theta_1,\theta_2)^\mathsf{T}$ be a pair of fixed-effect parameters, let $\hat{\theta}$ be their estimated values, and let $\widehat{V}$ be the estimated $2\times2$ covariance matrix of those estimates. An approximate $100(1-\alpha)\%$ joint Wald confidence region is the set of parameter pairs satisfying $$ (\theta-\hat{\theta})^\mathsf{T}\widehat{V}^{-1}(\theta-\hat{\theta})\leq \chi^2_{2,1-\alpha}, $$ where $\chi^2_{2,1-\alpha}$ is the $1-\alpha$ quantile of a chi-square distribution with 2 degrees of freedom.

The covariance matrix is $$ \widehat{V}= \begin{pmatrix} \widehat{\operatorname{Var}}(\hat{\theta}_1) & \widehat{\operatorname{Cov}}(\hat{\theta}_1,\hat{\theta}_2)\\ \widehat{\operatorname{Cov}}(\hat{\theta}_1,\hat{\theta}_2) & \widehat{\operatorname{Var}}(\hat{\theta}_2) \end{pmatrix}. $$ The off-diagonal entries are essential: setting them to zero would impose zero estimated correlation and generally give the wrong ellipse. For a pair selected from a larger fixed-effect vector, use the corresponding $2\times2$ block of its estimated covariance matrix. This block represents the pair's marginal estimation covariance, including the effects of estimating the other model parameters; it is not the covariance conditional on those other parameters being known.

Assumptions and the Ellipse

The Wald region relies on a local approximation. The fitted model should be identifiable near the estimate, the parameters should be away from problematic boundaries, and the estimator should be approximately bivariate normal with covariance estimated consistently. In likelihood-based fitting, this approximation is commonly motivated by regular large-sample likelihood theory. The covariance matrix must also be positive definite so that its inverse exists. These conditions are approximations in practical NLME analyses, not guarantees supplied merely by software reporting a standard error.

The cutoff for a 95% joint region is $\chi^2_{2,0.95}=5.991$, not $1.96^2$. The latter cutoff is associated with a single standardized normal coordinate, whereas the joint region concerns two coordinates at once.

Justification: Under the approximation $\hat{\theta}\sim N_2(\theta,V)$, the standardized vector $V^{-1/2}(\hat{\theta}-\theta)$ has approximately two independent standard normal components. The sum of their squared values therefore has a chi-square distribution with 2 degrees of freedom. Replacing $V$ with its estimate gives the Wald statistic, so values no greater than $\chi^2_{2,1-\alpha}$ define the approximate confidence region. Equivalently, under repeated sampling, this region contains the true pair in approximately $100(1-\alpha)\%$ of samples when the approximation holds.

The ellipse is centered at $\hat{\theta}$. Its orientation and elongation depend on the covariance, while its overall size also depends on the two standard errors. A strong correlation produces a tilted, elongated ellipse: changes in one parameter estimate tend to accompany changes in the other. A zero correlation gives an axis-aligned ellipse, but it is still not generally a rectangle.

Worked Example: Checking a Pair Against a 95% Wald Region

Suppose a hypothetical NLME fit estimates two dimensionless fixed effects, $\theta_1=\log(CL_{\mathrm{typ}}/(1\ \mathrm{L/h}))$ and $\theta_2=\log(V_{\mathrm{typ}}/(1\ \mathrm{L}))$. The estimates are $\hat{\theta}_1=1.20$ and $\hat{\theta}_2=2.10$, with standard errors 0.12 and 0.10 and estimated correlation $-0.60$. These are hypothetical values for illustration.

The covariance is $(-0.60)(0.12)(0.10)=-0.0072$, so the estimated covariance matrix is $$ \widehat{V}= \begin{pmatrix} 0.0144 & -0.0072\\ -0.0072 & 0.0100 \end{pmatrix}. $$ Its determinant is $0.0144(0.0100)-(-0.0072)^2=0.00009216>0$, and its diagonal entries are positive, so it is positive definite.

For a candidate pair $(1.35,1.98)$, the differences from the estimates are $0.15$ and $-0.12$, or 1.25 and $-1.20$ standard errors. In terms of standardized differences $x$ and $y$, the quadratic form for correlation $\rho$ is $$ q=\frac{x^2-2\rho xy+y^2}{1-\rho^2}. $$ Substituting $x=1.25$, $y=-1.20$, and $\rho=-0.60$ gives $$ q=\frac{1.25^2-2(-0.60)(1.25)(-1.20)+(-1.20)^2}{1-(-0.60)^2} =\frac{1.2025}{0.64}=1.879. $$ Since $1.879<5.991$, this candidate pair lies inside the approximate 95% joint region. This means it is not excluded by this Wald region at the corresponding approximate level; it does not assign a 95% probability to the fixed-effect pair.

Joint Regions Are Not Separate Marginal Intervals

A 95% marginal confidence interval for $\theta_1$ concerns that coordinate alone, and a separate 95% interval for $\theta_2$ concerns the other coordinate alone. Their Cartesian product is a rectangle. It does not, in general, have 95% joint coverage. If the two estimates were independent and both intervals had exactly 95% coverage, the probability that both intervals covered would be $0.95^2=0.9025$. With correlation, the rectangle's joint coverage changes, but two separate 95% labels still do not make it a 95% simultaneous region.

Worked Example: A Pair Inside Both Intervals but Outside the Ellipse

Use the hypothetical estimates and covariance from the previous example. The usual approximate 95% marginal intervals use $1.96$ standard errors. For $\theta_1$, the interval is $1.20\pm1.96(0.12)$, or $(0.9648,1.4352)$. For $\theta_2$, it is $2.10\pm1.96(0.10)$, or $(1.904,2.296)$.

Consider the pair $(1.416,2.280)$. It is inside both intervals: its first coordinate is $1.80$ standard errors above 1.20, and its second coordinate is also $1.80$ standard errors above 2.10. But its joint Wald statistic is $$ q=\frac{1.80^2-2(-0.60)(1.80)(1.80)+1.80^2}{1-(-0.60)^2} =\frac{10.368}{0.64}=16.20. $$ Because $16.20>5.991$, the pair lies outside the approximate 95% joint ellipse, despite lying inside each separate 95% marginal interval. The negative correlation is important here: a pair with both coordinates above their estimates is in a direction that is relatively far from the tilted ellipse.

Thus the intervals answer two separate coordinate questions, while the ellipse answers a question about the pair. Neither statement contradicts the other.

Constructing a Simultaneous Rectangle

A rectangle can be made simultaneous by choosing wider marginal intervals. One simple approach is Bonferroni adjustment: for a desired joint error rate $\alpha$, make each of the two marginal intervals have error rate at most $\alpha/2$. The probability that either interval misses is then no greater than the sum of the two miss probabilities, which is at most $\alpha$. This guarantees at least $1-\alpha$ joint coverage under the marginal coverage approximations, regardless of the correlation. It is often conservative compared with the ellipse.

Worked Example: Bonferroni Simultaneous Intervals

For the same hypothetical pair, target at least 95% simultaneous coverage. Use a 97.5% two-sided marginal interval for each parameter, whose normal critical value is approximately $z_{0.9875}=2.241$. The first interval is $$ 1.20\pm2.241(0.12)=1.20\pm0.2689=(0.9311,1.4689). $$ The second is $$ 2.10\pm2.241(0.10)=2.10\pm0.2241=(1.8759,2.3241). $$ Each interval has approximate 97.5% marginal coverage. By the union bound, the probability that either misses is at most $0.025+0.025=0.05$, so the rectangle has at least approximately 95% simultaneous coverage. The Bonferroni guarantee does not use the correlation, whereas the Wald ellipse uses the covariance to represent the joint uncertainty's direction.

Diagnostics, Alternatives, and Limitations

Before interpreting an ellipse, check that the covariance estimate is finite, symmetric to numerical tolerance, and positive definite. Verify that the standard errors and covariance come from the same fitted model and parameterization. If the covariance matrix is nearly singular, the ellipse may be extremely elongated and its inverse unstable; this can reflect strong dependence, weak identification, or numerical problems. Also compare the estimated correlation and ellipse with the model's parameter scales and the fitted likelihood behavior.

A normal-based ellipse can be misleading when likelihoods or sampling distributions are markedly skewed, when estimates approach constraints, or when parameters are weakly identified. The introductory material on profile likelihood and bootstrap intervals provides alternatives for examining uncertainty beyond a local normal approximation. For a pair, a likelihood-based joint region can be formed by comparing the maximized log likelihood with the log likelihood at each fixed candidate pair, reoptimizing nuisance parameters, and using a two-degree-of-freedom likelihood-ratio cutoff under regularity conditions. A bootstrap can instead examine the empirical distribution of the paired estimates; a joint contour should preserve their dependence rather than treating each coordinate independently.

These alternatives do not automatically solve every problem. A profile region still depends on reliable optimization and suitable likelihood approximation, while a bootstrap region depends on a defensible resampling scheme and enough successful replicates to characterize the joint distribution. If the paired estimates show strong skewness or a curved cloud, a single ellipse may be a poor summary even when the marginal standard errors look reasonable.

A common mistake is to take two 95% marginal intervals, call their combined rectangle a “95% joint confidence region,” and then interpret every pair inside it as jointly supported at that level. The correction is to state what each method covers: use a joint Wald ellipse or a simultaneous construction such as Bonferroni for a joint claim, and label separate intervals as marginal. In either case, confidence coverage describes repeated-sampling performance under the stated model and approximation; it is not the probability, after observing the data, that a fixed parameter value lies in the reported set.

Key takeaway: A two-parameter Wald region uses the full covariance submatrix and a chi-square cutoff with 2 degrees of freedom. Its joint coverage is distinct from the coverage of either marginal interval, and its reliability depends on the normal and covariance approximations being adequate.

Check Your Understanding

Answer each question, then compare with the solutions below.

What cutoff is used for an approximate 90% joint Wald region for two parameters, and why are there 2 degrees of freedom?
Suppose the estimated correlation between two fixed-effect estimates is zero. Does that make the joint Wald region a 95% rectangle when using two separate 95% marginal intervals? Explain.
With two 95% marginal intervals and independent estimates, what is the probability that both intervals cover, assuming exact 95% coverage for each?
Name two conditions or diagnostics that matter when deciding whether a Wald ellipse is trustworthy.

Solutions. (1) Use $\chi^2_{2,0.90}=4.605$; the joint vector has two coordinates, so the squared standardized distance has a chi-square distribution with 2 degrees of freedom under the approximation. (2) No. Zero correlation makes the ellipse axis-aligned, but separate 95% intervals still have only $0.95^2=0.9025$ joint coverage under independence, not 95%. (3) $0.95\times0.95=0.9025$, or 90.25%. (4) Examples include checking that the covariance matrix is positive definite and that the estimates are not near a boundary; also assess whether the paired-estimate distribution is approximately normal and whether the covariance estimate is numerically stable.