Approximation Bias in Sparse Sampling Designs

Why Sparse Sampling Matters

A population model can have many individuals but still provide little information about each individual’s random effects when each contributes only one or two concentration measurements. That distinction matters for approximating the marginal likelihood: FOCE uses a conditional mode and a local linearization, while Laplace uses the mode and local curvature of the individual log posterior. With sparse data, the observations may weakly locate the mode, and a small change in the local approximation can materially change an individual’s likelihood contribution.

Sparse sampling does not automatically make either approximation biased. The issue is conditional: the amount and timing of information per individual, the model’s nonlinearity over the plausible range of random effects, and the adequacy of the local approximation all matter. A precise population-level fit cannot make an individual mode well determined when that individual’s observations contain little information about the relevant random-effect direction.

Information, Conditional Modes, and Local Approximations

For clarity, suppose an individual has \(m_i\) observations, a model prediction vector \(f_i(\eta_i)\), and conditionally normal residual errors with covariance \(R_i\). Assume random effects are normally distributed with covariance \(\Omega\), observations are conditionally independent across individuals, and the model is smooth in \(\eta_i\). Up to terms that do not depend on \(\eta_i\), the individual negative log posterior is

The prior term penalizes random effects far from zero. The data term measures discrepancy between observed and predicted concentrations. A sparse individual can still have a unique mode because the prior contributes curvature, even when the data do not identify every component. In that case, the mode may be numerically stable while being largely driven by the prior rather than by that individual’s observations.

A useful local information description comes from linearizing \(f_i\) at a point and writing its Jacobian as \(J_i\). Under the stated normal model, the local negative-log-posterior curvature is approximately

The second term is the information supplied by the observations in random-effect directions. With few samples, or samples whose sensitivities point in similar directions, this matrix may have small eigenvalues. Then uncertainty is large along the corresponding combinations of random effects. The prior stabilizes the calculation but does not create subject-specific data information.

The exact marginal likelihood contribution integrates over random effects. FOCE replaces the nonlinear prediction locally by a linear approximation around the conditional mode and obtains a Gaussian-based contribution. Laplace instead approximates the integrand around the mode using its local Hessian. For an interior mode with positive-definite Hessian \(H_i=\nabla^2 g_i(\hat{\eta}_i)\), the basic Laplace form is proportional to

where \(q\) is the dimension of \(\eta_i\); normalizing constants from the residual and random-effects densities must also be included for a full likelihood. Thus the mode affects the height of the integrand, while curvature affects the volume around it. Under sparse sampling, a mode can move or its curvature can change substantially when the approximation point or model sensitivity changes.

How Sparse Data Change Likelihood Contributions

In a nonlinear concentration model, the prediction’s sensitivity to a random effect can vary with sampling time and with the random effect itself. A single concentration may constrain one combination of random effects but leave another nearly unconstrained. Two observations at nearly identical times may add less information than two observations at times where the prediction responds differently to clearance and volume effects. The number of observations alone is therefore not a complete measure of informativeness.

Sparse sampling can affect approximate contributions in two related but distinct ways. First, the conditional mode can be weakly determined: small changes in the data or model may shift it, or the mode may be pulled toward zero by the random-effects distribution. Second, the local approximation can be inaccurate if the prediction changes substantially over the range of random effects that contributes to the integral. FOCE depends on local linearization; Laplace depends on local curvature. A well-computed mode does not by itself ensure either local approximation is adequate.

The difference between curvature-based contributions can be seen without confusing it with the mode height. If near the mode the negative log integrand is quadratic, its local integral is proportional to \(H^{-1/2}\) in one dimension. A sparse nonlinear model may have appreciably different local curvature under two approximations.

Derivation: Why the Curvature Factor Appears

The Laplace curvature adjustment follows by expanding the negative log integrand near an interior mode. If \(g\) is twice differentiable, \(\nabla g(\hat{\eta})=0\), and \(H=\nabla^2g(\hat{\eta})\) is positive definite, then a second-order Taylor approximation gives

Substituting this quadratic form into the integral of \(\exp\{-g(\eta)\}\) separates out \(\exp\{-g(\hat{\eta})\}\). The remaining integral is a centered Gaussian integral, which equals \((2\pi)^{q/2}|H|^{-1/2}\). Consequently, a larger determinant of curvature means a narrower local region and a smaller approximate integral, all else equal. This justification requires an interior mode and positive-definite local curvature; it does not establish that the quadratic approximation is accurate far from the mode.

Sparse designs can make the distinction especially visible. When the data provide little curvature in a random-effect direction, the prior may dominate that direction. If the likelihood is also noticeably nonlinear across the posterior range, the second-order expansion can represent only a small neighborhood of the mode. FOCE’s conditional linearization and Laplace’s quadratic expansion are then sensitive to the local geometry, not simply to the number of subjects in the study.

Diagnostics, Alternatives, and Limitations

A sparse-sampling concern should be assessed through information and approximation behavior, not inferred from sample count alone. Useful checks include:

• Inspect the design sensitivity. Compare model sensitivities at actual sampling times. Similar sensitivity rows or weak response to a random effect indicate that some directions may be poorly informed.
• Inspect conditional modes and curvature. Large changes in \(\hat{\eta}_i\) under small data perturbations, or a nearly singular \(H_i\), suggest weakly determined directions. A positive prior contribution can keep \(H_i\) nonsingular without making the data informative.
• Check the local approximation. Compare predictions near the mode with the model’s nonlinear predictions over a plausible range around it. Large discrepancies warn that local linear or quadratic behavior may not represent the integration region.
• Compare individual contributions under reasonable approximations. Large discrepancies can flag sensitivity, but they do not by themselves identify which approximation is closer to the exact integral.

The main alternatives have different strengths. FO is a first-order approximation around the population mean and can be computationally simple, but it does not use the same individual-specific conditional re-centering as FOCE. FOCE re-centers at the individual conditional mode and linearizes there, which can help when subject-specific deviations matter; a poorly informed mode or strong local nonlinearity can still affect it. Laplace uses the mode and Hessian and can better represent local curvature, but it is also a local approximation. No one method can restore information absent from the observations. More informative sampling times, additional measurements, or a scientifically justified simpler random-effects structure may address that design limitation more directly.

A common mistake is to conclude that a small number of samples necessarily makes Laplace biased and that FOCE must therefore be preferred. The reasoning fails because sparse sampling can weaken mode identification, but approximation error depends on the shape of the integrand as well as the design. Laplace may be accurate for a nearly quadratic integrand with sparse data, while either approximation can be inaccurate for a strongly nonquadratic integrand. Conversely, a substantial difference between FOCE and Laplace is a warning to investigate, not proof that one estimate is correct.

These checks also have limits. Conditional modes are empirical-Bayes quantities and are not direct observations of true individual effects. Their apparent stability can be produced by strong prior shrinkage. Approximation comparisons are not a substitute for evaluating the marginal integral more accurately when that is feasible, and a richer integration method cannot compensate for an intrinsically uninformative design.