Why Confidence-Interval Mistakes Matter
An NLME parameter confidence interval summarizes uncertainty about a population model parameter, such as typical clearance, a covariate effect, or a random-effect variance. Its interpretation depends not only on the final two endpoints but also on how the data were resampled, which model fits were retained, and the scale on which the interval was calculated. A plausible-looking interval can still be misleading if any of those steps violates the structure of the model or the assumptions behind the method.
Write the repeated measurements for subject \(i\) as \(y_{ij}\), where \(i=1,\ldots,N\) indexes subjects and \(j\) indexes observations within a subject. A typical NLME model represents an observation as
Here \(f\) is the structural model, \(t_{ij}\) denotes the observation time or design variables, \(\phi_i\) is a subject-specific parameter vector, \(\theta\) is the population parameter vector, \(b_i\) is a subject-level random effect, and \(\epsilon_{ij}\) is residual error. The exact assumptions about random effects and residuals depend on the fitted model. The key feature for the mistakes discussed here is that several observations may belong to the same subject and share the same \(b_i\). They are therefore not generally independent sampling units.
Mistake 1: Resampling Observations Instead of Subjects
In a subject bootstrap, the sampling unit is ordinarily the independent subject, not each recorded concentration or response separately. One resamples subjects and carries along each selected subject’s complete record: its observation times, outcomes, and relevant covariates. The NLME model is then refitted to each resampled data set. This preserves the within-subject pattern that the model attributes to shared subject-level effects.
If individual observations are sampled independently, measurements from one original subject can be split across several pseudo-subjects, while measurements from different subjects can be combined into one pseudo-subject. This destroys the original clustering. It can make the resampled data look as though they contain more independent information than they do, often producing intervals that are too narrow. The direction and size of the error are not guaranteed in every model, but the resampling scheme no longer represents the stated subject-sampling process.
Worked Example: Keeping Each Subject’s Record Together
Suppose a hypothetical study has four subjects, each with three concentration measurements. For illustration, assume the fitted subject-level clearance summaries are 4, 6, 8, and 10 L/h. The target analysis is a subject bootstrap followed by refitting the NLME model; these summaries are used only to show what resampling the subject units means.
A possible bootstrap draw of four subjects, with replacement, is subject 1, subject 2, subject 2, and subject 4. The corresponding illustrative mean of the subject summaries is
For the actual NLME bootstrap replicate, all three records for each selected subject are copied together, and the population model is refitted. Subject 2’s complete record appears twice in this replicate; subject 3’s record does not appear. By contrast, drawing 12 isolated measurements from the 12 original rows could select one time point from subject 1 and another from subject 3 and assign them to the same artificial subject. That data set does not preserve the original subject-level sampling structure.
The bootstrap unit should match the independent sampling unit. If the design samples independent subjects, resample subjects. In a hierarchical design with additional independent sampling levels, the resampling scheme must respect those levels and the target population. A single measurement per subject is a special case in which subject-level and observation-level resampling coincide.
Mistake 2: Treating Failed Fits as Valid Estimates
A bootstrap replicate is useful for an interval only if its fitted parameter estimate is a defensible estimate from that replicate. An optimizer can stop without finding an adequate optimum; a software status labelled “converged” does not, by itself, establish that the likelihood has been maximized or that the estimate is reliable. Warning signs include a large final gradient, parameters stuck at implausible bounds, a non-positive-definite Hessian when one is required for the reported calculation, or estimates that change substantially under reasonable changes in starting values.
A failed fit must not be entered as though its last parameter value were a valid estimate. Nor should it be replaced by zero, the original-data estimate, or a convenient boundary value. Such substitutions create artificial features in the bootstrap distribution and can move its quantiles in arbitrary directions. However, simply discarding every replicate that presents difficulty is not automatically safe: if failures are common or related to the resampled data, the retained estimates may represent a selected subset of replicates rather than the intended sampling distribution.
Worked Example: Auditing Bootstrap Fits Before Taking Quantiles
In a hypothetical subject bootstrap with 1,000 replicates for typical clearance, 912 fits meet prespecified convergence and diagnostic checks. Another 38 have a nominal convergence code but unacceptable gradients or unstable estimates, and 50 terminate without a usable optimum. Thus, the fraction that passes the checks is
Suppose the 2.5th and 97.5th empirical percentiles of the 912 checked estimates are 4.2 and 7.8 L/h. These are the percentile endpoints among the fits that passed the checks; the arithmetic does not establish that they form a calibrated 95% interval. The 88 replicates that did not pass amount to 8.8% of the attempted fits, enough to investigate rather than silently ignore.
The analyst should review the failed replicates and their data, test whether reasonable starting values or optimizer settings resolve numerical problems, and assess whether failures are concentrated in particular data configurations or parameter regions. If the instability persists, report the failure rate and its implications, and avoid presenting the retained-fit percentiles as if all 1,000 replicates had produced valid estimates. A high failure rate can itself indicate weak information, practical nonidentifiability, or a model that is difficult to fit to plausible resamples.
A useful audit records the attempted, successful, and diagnostically acceptable replicate counts; the criteria used to classify fits; parameter estimates at bounds; and the distribution of estimates from accepted fits. Inspecting only the final interval hides whether its endpoints are based on a stable, representative set of refits.
Mistake 3: Ignoring the Parameter Scale
Many NLME parameters are constrained: clearance and variance components must be positive, for example. Software may estimate a transformed parameter such as \(\eta=\log(CL)\), for which any real value maps to a positive clearance \(CL=\exp(\eta)\). An interval calculated on the transformed scale must be returned to the original scale by transforming both endpoints. In general, the transformed interval is asymmetric around the estimate on the original scale.
This follows directly from the order-preserving property of the exponential function. If an interval for \(\eta\) is \([L_\eta,U_\eta]\), then exponentiating each endpoint gives \([\exp(L_\eta),\exp(U_\eta)]\) for \(CL\). A common incorrect shortcut is to calculate a symmetric Wald interval on the transformed scale and then use its standard error as though it were an original-scale standard error.
Worked Example: Back-Transforming a Clearance Interval
Suppose a hypothetical model estimates \(\eta=\log(CL)=\log(5)=1.609\), with standard error 0.200 on the log scale. Under the approximate normal assumption used for a Wald interval, the 95% interval for \(\eta\) is
Exponentiating the endpoints gives the interval for clearance:
The interval is asymmetric around 5 L/h, as expected for a positive parameter obtained by exponentiation. For comparison, a first-order delta-method calculation gives an approximate original-scale standard error of \(CL\times SE(\eta)=5(0.200)=1.00\) L/h. A symmetric original-scale Wald calculation would then give \(5\pm1.96(1.00)=[3.04,6.96]\) L/h. This is a different approximation, not the back-transformed interval above. The analyst must state which scale and interval method were used; mixing the transformed estimate with an incorrectly interpreted standard error is not valid.
The local justification for the delta-method standard error is that for a smooth transformation \(h\), the approximate variance of \(h(\hat{\eta})\) is \([h'(\eta)]^2\operatorname{Var}(\hat{\eta})\), evaluated near the estimate. For \(h(\eta)=\exp(\eta)\), \(h'(\eta)=\exp(\eta)\), giving the approximation used above. It is a local approximation; it does not make the sampling distribution symmetric on the original scale.
Mistake 4: Reporting Ordinary Wald Limits at a Boundary
An ordinary Wald interval has the form estimate \(\pm\) a normal critical value times its standard error. Its usual justification relies on an approximately normal estimator centered at the true parameter and on regular behavior near that parameter. A parameter at a boundary, such as a random-effect variance constrained to be nonnegative and estimated as zero, does not generally satisfy that interior approximation. The likelihood can be asymmetric, the estimate can pile up at zero, and the usual standard error may be unstable or undefined.
Worked Example: A Negative Wald Limit for a Variance
Suppose a hypothetical random-effect variance estimate is \(\hat{\omega}^2=0.0025\) in squared log-clearance units, with reported standard error 0.0040 in the same units. The naive 95% Wald limits are
A negative variance is not an allowable model value. Replacing the lower endpoint by zero does not restore the normal approximation or establish that \([0,0.01034]\) has 95% coverage. In this example, the estimate is close to the boundary relative to its standard error, so the ordinary symmetric Wald interval is especially questionable. An interval based on a suitable profile likelihood or a carefully designed bootstrap may better reflect the constraint and asymmetry, but each method still requires its own assumptions and diagnostics.
The same warning applies when a parameter estimate is exactly on a boundary. A zero variance estimate is not an ordinary interior estimate with an inconvenient negative lower limit; it signals that the regular approximation underlying the standard Wald construction may fail. Boundary-aware methods do not guarantee a useful interval if the data contain little information, so the likelihood shape and the estimation behavior still need inspection.
Diagnostics, Alternatives, and Limitations
Before interpreting an interval, check that the reported parameter scale and units are clear, that constraints are respected, and that the interval method’s assumptions are plausible. For bootstrap intervals, inspect the replicate distribution and fit diagnostics, not just the endpoints. For profile intervals, examine whether the profile is adequately optimized and whether its threshold crossings are well determined. For Wald intervals, consider whether the estimate is sufficiently far from boundaries and whether the local normal approximation is credible.
| Approach | Potential advantage | Important caution |
|---|---|---|
| Wald interval | Direct and inexpensive when the estimator is approximately normal and interior. | Can be misleading with skewness, weak identification, or boundary estimates; scale matters. |
| Subject bootstrap | Can represent sampling variation without relying on a symmetric normal approximation. | Requires a suitable resampling unit and stable, diagnostically acceptable refits. |
| Profile-likelihood interval | Can reflect likelihood asymmetry and parameter constraints. | Requires reliable constrained optimization and a sufficiently informative profile. |
These methods answer uncertainty questions under different approximations; none automatically repairs a misspecified model, an inappropriate sampling design, or inadequate information. An interval for a population parameter also should not be confused with a prediction interval for a new subject or with the spread of individual random effects. Those are different targets.
Common mistake: “The software returned a 95% interval, so the endpoints are valid.” The correction is to identify the method, check its assumptions, verify the scale and constraints, and review diagnostics for the underlying fit or bootstrap replicates. Software output is a calculation, not a guarantee of calibration.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A study samples independent subjects and measures each subject repeatedly. Which unit should ordinarily be resampled for a subject bootstrap, and what must be carried along?
- Of 500 bootstrap fits, 450 pass prespecified checks. What percentage passed, and why is it not automatically safe to report percentiles from those 450 alone?
- A log-scale interval for \(\eta=\log(CL)\) is \([0.8,1.4]\). What is the corresponding clearance interval?
- Why does changing a negative Wald lower limit for a variance to zero not, by itself, produce a valid 95% interval?
- Name one diagnostic concern for a bootstrap refit and one alternative to an ordinary Wald interval.
Solutions: (1) Resample subjects and carry each selected subject’s full record, including its repeated outcomes and relevant design information. (2) \(450/500=0.90\), so 90% passed. Failures may be related to the resampled data or parameter region, making the retained fits a selected subset; investigate and report the failure pattern. (3) Exponentiate both endpoints: \([\exp(0.8),\exp(1.4)]\), approximately \([2.226,4.055]\) in the same clearance units used by the model. (4) The estimate is on a boundary where the ordinary normal approximation may fail; truncation changes the interval without correcting that approximation or establishing its coverage. (5) Examples include a large final gradient, unstable estimates under changed starting values, or estimates stuck at bounds; suitable alternatives include a subject bootstrap or a profile-likelihood interval, subject to their own assumptions and diagnostics.