Generating Population and Individual External Predictions

Prediction Targets in External Evaluation

An external prediction must specify whose outcome is being predicted and what information is available when the prediction is made. A prediction for a new subject using only baseline covariates and a planned dosing history is not the same as a forecast for a subject whose earlier concentration measurements are already available. The first uses no subject-specific outcome data; the second can use those permitted observations to update the subject’s random effects.

In either case, the external model is treated as frozen: its structural form, population parameter estimates, random-effect distribution, and residual model are not re-estimated from the external outcomes. Baseline covariates, dosing, and observation times must be supplied in the model’s defined units and form. For a forecast, only observations available before the forecast time may inform the subject-specific update.

These terms refer to different calculations. Setting random effects to zero gives a useful typical-subject curve, but in a nonlinear model it generally does not equal the population average. Likewise, an individual forecast is not simply a population prediction with an estimated random effect plugged in: uncertainty about that random effect can affect the forecast, particularly when the outcome is transformed, as with concentrations modeled on the log scale.

Model and Prediction Assumptions

Write a general NLME model for subject \(i\) as

Here \(Y_{ij}\) is the modeled response at time \(t_{ij}\), \(D_i\) denotes the dosing history, \(X_i\) denotes baseline covariates, \(\theta\) contains population parameters, and \(\eta_i\) is the subject’s vector of random effects with covariance matrix \(\Omega\). The residual error \(\varepsilon_{ij}\) follows the model’s specified residual distribution, conditionally on the subject effects and design information. The function \(f\) may be nonlinear in time, dose, covariates, or random effects.

For external prediction, the model and its parameter estimates are held fixed. A new subject’s random effects are drawn from the model’s population distribution; for an individual forecast, their distribution is updated using that subject’s prior observations. The calculations assume that the structural and error models are appropriate for the prediction target, that the dosing and covariate inputs have been harmonized correctly, and that the observations used for updating are genuinely available before the forecast. They also rely on the model’s random-effects distribution being a reasonable description of the subjects to whom predictions are applied.

Typical and Population-Marginal Predictions

For a typical-subject prediction, set \(\eta_i=0\) in the structural model:

This is a plug-in prediction at the center of the random-effects distribution. For a population-marginal prediction of the latent response, integrate the structural prediction over that distribution:

The integral is generally not equal to \(f(t,D_i,X_i;\widehat{\theta},0)\), because a nonlinear function of a random variable does not usually have expectation equal to the function evaluated at that variable’s expectation. If the target is a future measured response rather than the latent structural response, the residual distribution must also be included in the predictive expectation or distribution.

Empirical-Bayes Forecasts From Prior Observations

For an individual forecast, let \(d_i\) denote the subject’s permitted prior observations together with their times and relevant design information. The posterior random-effects density is proportional to the likelihood of those data conditional on the random effects, multiplied by the population random-effects density:

An empirical-Bayes point estimate, often the posterior mode or posterior mean, summarizes this distribution. A forecast for a future latent response should, when the target is its conditional expectation, average over the full posterior distribution:

For a future measured response, the residual-error model is included as well. In general, substituting the posterior mean of \(\eta_i\) into \(f\) produces a plug-in prediction, not necessarily the conditional expectation in this equation. For nonlinear functions, these quantities can differ.

Deriving the Gaussian Update

A simple log-concentration model shows how prior observations update an individual random effect. Suppose \[ \log C_i(t)=\log(12.0)-0.150t+\eta_i, \qquad \eta_i\sim N(0,0.250), \] and observed log concentrations satisfy \[ z_{ij}=\log C_i(t_{ij})+\varepsilon_{ij}, \qquad \varepsilon_{ij}\sim N(0,0.090), \] independently conditional on \(\eta_i\). Define each log-scale residual relative to the population prediction as \(r_{ij}=z_{ij}-[\log(12.0)-0.150t_{ij}]\). Then \(r_{ij}=\eta_i+\varepsilon_{ij}\).

The prior density contributes a quadratic term \(\eta_i^2/0.250\) to the negative twice log density, and the observation likelihood contributes \(\sum_j(r_{ij}-\eta_i)^2/0.090\). Collecting the coefficients of \(\eta_i^2\) and \(\eta_i\) gives a normal posterior with variance \(v_i\) and mean \(m_i\):

The posterior variance is smaller than the prior variance when observations provide information, and the posterior mean is pulled toward the observed residuals. This derivation assumes the stated Gaussian prior and independent Gaussian residuals with the specified variance. Different residual models or nonlinear observation models generally require different posterior calculations.

Choosing and Checking the Prediction

The appropriate calculation depends on the prediction target and the information set, not on which calculation gives a more favorable result. A typical-subject curve is useful for describing the model at zero random effect. A population-marginal prediction is appropriate when the target is an average over new subjects with the supplied covariates and regimen. An empirical-Bayes forecast is appropriate when the target is a particular subject and prior outcomes are explicitly available at the forecast time.

Before generating predictions, verify the following:

• Information timing: Confirm that every outcome used for an individual update precedes the forecast time. A later concentration cannot be used to forecast itself or an earlier concentration.
• Prediction scale: State whether the target is latent concentration, a conditional median, an arithmetic mean, or a future measured concentration. These can differ under lognormal models.
• Inputs and model: Check units, dose history, covariates, time origin, and the frozen parameter values. A unit mismatch can produce a plausible-looking but invalid curve.
• Random-effects assumptions: Check whether the assumed distribution and covariance are plausible for the prediction population. Posterior calculations may be misleading if those assumptions are poor.
• Numerical behavior: For models without a closed-form integral, compare numerical integration or simulation results for stability and check that predictions remain finite over the requested times.

Empirical-Bayes estimates are often shrunk toward the population mean, especially when a subject has few or noisy observations. Shrinkage is an implication of combining the observations with the population prior; it is not evidence that the individual truly has an average parameter value. Conversely, using a subject’s observed outcomes to update random effects is not the same as refitting the population model. The former is individual conditioning; the latter changes the model estimates and can compromise an external evaluation if it uses validation outcomes.

Alternatives and Limitations

A plug-in forecast evaluates the structural model at a point estimate such as the posterior mode or mean of \(\eta_i\). It is straightforward and can be useful as a point summary, but for nonlinear models it need not equal the posterior predictive mean and does not itself represent the remaining uncertainty. Integrating over the posterior random-effects distribution is more directly aligned with an expected-outcome target. Simulation from that posterior can approximate the integral when it is not available analytically, provided the simulation is adequately precise.

Baseline-only population predictions avoid individual outcome updating and are therefore suitable for a new subject before any postbaseline measurements are available. They cannot use subject-specific response information. Empirical-Bayes forecasts can be more individualized, but depend on the adequacy of the population prior and observation model; with sparse data, the forecast may remain close to the population prediction. Neither approach repairs model misspecification, poor external covariate support, incorrect dosing records, or an inappropriate residual model. Generating predictions is not itself evidence that they are accurate; comparison with external outcomes is a separate evaluation task.