Approximation Behavior With Rich Within-Subject Data

Why Rich Within-Subject Data Matter

A subject with many concentration measurements can provide more information about that subject’s random effects than a subject with only a few measurements. In the usual regular setting, the conditional distribution of the random effects then concentrates more tightly around its conditional mode. This precision is useful, but it does not automatically make every approximation to the marginal likelihood accurate: information depends on what was measured, and approximations can use local curvature differently.

The key distinction is between learning an individual effect more precisely and approximating the population likelihood more accurately. Rich observations can help with the first while leaving the second sensitive to nonlinear curvature, the location of the mode, and the residual model.

For clarity, first consider one random effect and independent additive residuals: \(y_{ij}=f(t_{ij};\theta,\eta_i)+\epsilon_{ij}\), with \(\epsilon_{ij}\sim N(0,\sigma^2)\). The negative log posterior, omitting constants that do not depend on \(\eta_i\), is

The first term is the random-effects penalty; the second measures the fit to that subject’s observations. If the model is locally close to linear in \(\eta_i\), the data contribution to curvature is approximately \(J_i^\mathsf{T}R_i^{-1}J_i\), where \(J_i\) is the matrix of derivatives of the mean predictions with respect to \(\eta_i\), and \(R_i\) is the residual covariance matrix. With independent residuals of variance \(\sigma^2\), \(R_i=\sigma^2 I\). Thus, observations add information when their predictions change with the random effects. Merely taking many samples does not ensure high information if their sensitivities are small or redundant.

How Repeated Measurements Sharpen a Conditional Mode

A useful local calculation treats the predictions as linear in a scalar random effect near the mode. Suppose \(y_j\) is approximately \(f_{0j}+g_j\eta+\epsilon_j\), where \(g_j\) is the local sensitivity to \(\eta\). With prior variance \(\omega^2\) and independent residual variance \(\sigma^2\), the posterior precision and local variance are

This formula shows the role of both sample count and design. If each new observation has similar nonzero sensitivity and independent residual error, each adds positive information. The variance decreases as information accumulates, though the rate depends on the sensitivity and residual scale. If measurements are highly correlated, the independent-error formula overstates their combined information; the covariance matrix must then represent that dependence.

A Concentration-Time Example

Consider a hypothetical one-compartment concentration profile with a random log-scale effect: \(f(t;\eta)=100\exp(-0.2t)\exp(\eta)\), with concentration in a stated concentration unit and time in hours. Let \(\eta\sim N(0,0.25)\) and let additive residual standard deviation be \(5\) concentration units. For observations that exactly follow the profile at the chosen evaluation point \(\eta_0=\log(1.1)=0.09531\), the fitted concentrations at times \(0,1,2,3\) hours are approximately \(110.0,90.06,73.74,60.37\). The derivative with respect to \(\eta\) equals the fitted concentration itself. Since the residual contribution to the gradient is zero there but the prior gradient is \(\eta_0/0.25\ne0\), this point is not the conditional mode.

At this exact fit, the residual terms are zero. The scalar curvature at the chosen evaluation point is therefore the prior precision plus the sum of squared sensitivities divided by the residual variance. With one measurement at each of the four times, the sum of squared fitted concentrations is approximately \(29{,}292\), giving

If the same four time points are each measured twice, with the same fitted profile and independent residuals, the data contribution doubles:

The calculation illustrates how repeated observations can increase local curvature at the chosen evaluation point when the model is sensitive to the random effect. It also illustrates why design matters: a sample at a time where the concentration is low may contribute less information about this log-scale effect than a sample where the fitted concentration is high. These inverse-square-root-curvature quantities describe the local curvature at that evaluation point, not conditional-mode uncertainty or guaranteed frequentist standard errors for every estimation procedure.

Why FOCE and Laplace Can Still Differ

FOCE evaluates a local linearization around a subject’s conditional mode. Laplace uses the mode and the Hessian of the negative log posterior there to approximate the integral over random effects. Their relationship is clearest in the scalar additive-error case. Write \(r_j=y_j-f_j(\eta)\), \(f'_j=\partial f_j/\partial\eta\), and \(f''_j=\partial^2 f_j/\partial\eta^2\). At the mode, the exact negative-log-posterior curvature is

A curvature based on the conditional linearization uses the squared first derivatives and does not include the residual-weighted second-derivative term. Consequently, even when the mode is located accurately, curvature-based likelihood contributions can differ. As the number of well-behaved observations grows, the leading information term often grows with sample size. Under regularity conditions, residual-weighted curvature may be smaller relative to that leading term, making the local quadratic approximation more useful. But this is a conditional expectation about a regular design and model, not a guarantee based on sample count alone.

Assumptions, Diagnostics, and Limits

The preceding information calculations rely on a correctly specified conditional mean and residual model, a positive-definite local Hessian, and observations that contribute valid information about the random effects. The repeated measurements are treated as conditionally independent in the examples unless a covariance structure is explicitly included. If residual correlation, assay effects, or time-dependent variance are present but omitted, the apparent information gain from additional samples may be overstated.

When reviewing an analysis with rich within-subject data, examine whether:

• the conditional modes and their Hessians are stable under reasonable numerical settings;
• the sampling times cover regions where the predictions are sensitive to the random effects of interest;
• the local Hessian is positive definite and not nearly singular;
• residual patterns suggest that the assumed residual model is inadequate; and
• FOCE and Laplace objective contributions differ substantially for subjects with particular sampling patterns or large residuals.

A dense cluster of measurements in a low-sensitivity region may add little information. Likewise, repeated measurements with unmodeled correlation do not act like the same number of independent observations. If the conditional mode remains poorly determined, a small nominal standard error from an assumed independent-error calculation should not be taken as evidence of real precision.

The main alternatives answer related but distinct questions. FO expands around the population random-effects value, rather than centering the linearization on each subject’s conditional mode; it can therefore be less responsive when a subject’s well-informed mode is far from zero. FOCE uses conditional modes and local linearization. Laplace additionally uses local posterior curvature in approximating the integral. Rich subject-level data can help FOCE and Laplace locate a concentrated mode, while not rescuing an FO expansion that is poorly centered. None of these local approximations corrects a misspecified structural or residual model.

The concentration example used one random effect. With several random effects, the curvature is a matrix, and a large number of observations does not ensure that every direction is informed: the sensitivity matrix may have nearly dependent columns. Local curvature also describes behavior near one mode only. It cannot by itself establish that the approximation is reliable over the whole region contributing to the marginal likelihood.