Two Predictive Targets
A predictive check asks whether data simulated from a fitted model reproduce features of the observed data. The key choice is what is allowed to vary between the observed and replicated data. In a population-level check, each simulated individual receives a newly drawn individual effect from the model’s population distribution. In a conditional check, the simulated records retain that individual’s estimated effect, while new residual errors generate the replicate observations.
This distinction matters in repeated-dose pharmacokinetics. A population-level check asks whether the model can reproduce concentration patterns across a population with the observed covariates, dosing histories, and sampling schedules. A conditional check asks whether the model can reproduce records for these particular individuals, given their estimated individual effects. The two checks can disagree without contradiction: they assess different predictive targets.
For example, in a repeated-dose model, the concentration prediction depends on dose amounts and times, sampling times, covariates, population parameters, and individual effects. The design is ordinarily held to the observed design for both checks: the same number of individuals, dose histories, covariate values, and sampling schedules. The difference is whether individual effects are redrawn or retained. Residual errors are generated anew in both types of check, according to the fitted residual-error model.
What Each Check Includes
Write the model for a concentration record as
Here \(f_{ij}\) is the model-predicted concentration at record \(j\) for individual \(i\), \(\theta\) denotes population parameters, \(\Omega\) is the random-effects variance-covariance matrix, and \(\epsilon_{ij}\) is residual error. In practice, the residual-error distribution may be additive, proportional, combined, or otherwise specified; a predictive check must use the fitted model’s chosen form. The normal random-effects expression is illustrative of a common assumption, not a requirement for every model.
A population-level replicate varies both \(\eta_i\) and \(\epsilon_{ij}\). It therefore includes between-individual variation represented by the random-effects model, as well as within-individual residual variation. A conditional replicate fixes \(\eta_i=\hat{\eta}_i\), so it includes residual variation around the trajectory implied by that estimated effect, but it does not reproduce a new draw from the population distribution of individual effects.
This distinction can be expressed through the law of total variance. At a fixed sampling time and design, let \(C\) be a concentration with conditional mean \(m(\eta)=E(C\mid\eta)\) and conditional variance \(v(\eta)=\operatorname{Var}(C\mid\eta)\). Then
The first term represents variation in expected concentrations across individuals; the second represents average variation within the conditional response distribution. A population-level check reflects both terms. A conditional check at a fixed \(\hat{\eta}_i\) focuses on the second term for the selected individual. This is a conceptual decomposition: with nonlinear models, the size and pattern of the terms can depend on time, dose history, and covariates.
Worked Example: A Population-Level Trough Check
Suppose a hypothetical study gives 100 individuals the same repeated intravenous dose schedule, with a trough concentration recorded for each individual after the fourth dose. The observed troughs include 18 concentrations below 1.0 mg/L. In 1,000 population-level replicate data sets, the median number below 1.0 mg/L is 11, and the central 90% of replicate counts ranges from 6 to 17.
For every replicate data set, the check draws a new individual effect for each of the 100 individuals, applies the recorded dose and sampling design, and generates residual errors. The summary is the count below the stated threshold. The observed count is \(18/100=0.18\), or 18%. The replicate median is \(11/100=0.11\), or 11%, and the upper end of the stated reference range is \(17/100=0.17\), or 17%.
The observed count is just above the central 90% range in this hypothetical example. That suggests the fitted population model may underpredict the number of low troughs for this design. It does not identify the cause by itself: the random-effects distribution, a dose or covariate relationship, the structural concentration model, or the residual model could contribute. Nor does the result establish that the model is wrong with a particular probability; it is a discrepancy between an observed summary and its simulated reference distribution.
Conditional Checks for Individual Records
Conditional checks are useful when the question concerns the adequacy of predicted records for the individuals actually observed. For each individual, an estimated effect—often an empirical Bayes estimate—is obtained from the fitted model. Replicate concentrations are then generated with that estimate held fixed, while residual errors vary. A conditional check can reveal, for example, whether repeated-dose peaks or troughs for these individuals are systematically missed after accounting for their estimated differences in disposition.
The interpretation requires care. An empirical Bayes estimate is inferred using that individual’s observed data and is not known without error. Holding it fixed treats the estimate as if it were the true effect for the purpose of the check. When individual information is limited, shrinkage pulls estimates toward the population mean, so a conditional check may not represent the individual effects perfectly. The earlier material on eta shrinkage explains why estimated effects should not be interpreted as error-free individual truths.
Worked Example: A Conditional Residual-Pattern Check
Consider 12 hypothetical individuals receiving repeated doses, with two trough samples per individual, for 24 records in total. After fitting the model, the observed data contain 7 records whose absolute standardized residual exceeds 1.96. In 1,000 conditional replicate data sets, each individual’s estimated effect is retained and the residual errors are simulated anew. The median number exceeding 1.96 is 1, and the central 90% range is 0 to 3.
The observed proportion is \(7/24=0.2917\), or about 29.2%. The simulated median proportion is \(1/24=0.04167\), or about 4.17%; the upper end of the reference range is \(3/24=0.125\), or 12.5%. Thus, the observed number of unusually large residuals is greater than the range typically produced by these conditional replicates.
This result points to a potential problem in the conditional observation model or in the conditional concentration trajectories at the sampled records. It does not specifically demonstrate that the population distribution of individual effects is inadequate, because the conditional check held those effects fixed. Before attributing the pattern to residual error, also inspect whether exceedances cluster at a particular dose interval or sampling time: a time-related structural mismatch can appear as a residual problem.
Choosing and Comparing the Checks
Choose the check from the scientific question, not from which result looks more favorable. If the target is performance for new individuals from the modeled population under a defined design, use a population-level check. If the target is the conditional fit to the individuals and records in hand, use a conditional check. Both can be informative, and comparing them can help locate which component of the model deserves further attention.
| Feature | Population-level check | Conditional check |
|---|---|---|
| Individual effects | Redrawn from the fitted population distribution | Held at each observed individual’s estimated effect |
| Residual errors | Generated anew | Generated anew |
| Main predictive target | New individuals under the specified design | Observed individuals, conditional on estimated effects |
| Can assess population heterogeneity? | Yes, as represented by the fitted random-effects distribution | Not directly, because effects are not redrawn |
| Important limitation | May combine several sources of mismatch in a single discrepancy | Plug-in estimated effects can make the conditional fit appear more secure than it is |
A useful comparison is to examine the same concentration summary under both targets. A mismatch in the population-level check that is not present conditionally may indicate that the model’s population variation or distribution of individual effects needs attention. A mismatch in both may suggest a problem shared by the conditional trajectories or residual model. These are diagnostic clues, not unique diagnoses: different misspecifications can produce similar patterns.
Worked Example: Interpreting a Difference in Variance
At a specified trough time in a hypothetical repeated-dose setting, suppose the model implies that the variance of individual conditional mean troughs is \(49\;(\mathrm{mg/L})^2\), and the residual variance at that time is \(9\;(\mathrm{mg/L})^2\) for every individual. The observed trough standard deviation is 8.2 mg/L.
Under these simplifying assumptions, the population variance is
The corresponding population standard deviation is \(\sqrt{58}=7.616\) mg/L, approximately 7.62 mg/L. A conditional replicate for a fixed individual effect has variance \(9\;(\mathrm{mg/L})^2\), and therefore standard deviation \(\sqrt{9}=3.00\) mg/L around that individual’s conditional mean.
The observed standard deviation, 8.2 mg/L, is larger than the model-implied population standard deviation of about 7.62 mg/L, so the observed spread merits investigation in a population-level check. Comparing 8.2 mg/L directly with the conditional standard deviation of 3.00 mg/L would be inappropriate: the latter excludes between-individual variation. This calculation is a variance-decomposition illustration, not a substitute for comparing the full observed and simulated distributions.
Assumptions, Diagnostics, and Limitations
The interpretation of either check depends on how well the simulated data correspond to the intended target. At minimum, the model used to simulate must specify the concentration trajectory, random-effects distribution, residual-error distribution, and relevant dose and observation design. The observed and replicated summaries must be defined in the same way. If the goal is to compare the observed cohort under its actual design, retain its covariates, dose histories, and sampling times in the replicates; otherwise, differences in design may be mistaken for model mismatch.
A practical diagnostic workflow is to make the target explicit, select a summary that addresses it, compare the observed summary with replicated summaries, then investigate the records or model components that could explain a discrepancy. For repeated-dose data, summaries might distinguish troughs from peaks or describe concentrations at selected post-dose times. A single overall summary can conceal a time-localized pattern, while excessively narrow summaries can be unstable in small data sets.
Common failure modes include:
- Redrawing effects for a conditional question. This changes the target to a population-level prediction and adds between-individual variation that the conditional question does not ask about.
- Holding effects fixed for a population question. This removes the modeled variation across new individuals and cannot establish whether the population distribution reproduces the observed spread.
- Treating conditional estimates as known true effects. Estimated effects have uncertainty and may be shrunk toward the population mean, especially when individual data are sparse.
- Attributing every mismatch to one model component. A predictive discrepancy identifies a feature the fitted model does not reproduce well; it usually does not uniquely identify its cause.
- Changing the design between observed and replicate data. Different dose timing or sampling can change concentrations even when the model is unchanged, making the comparison hard to interpret.
Both checks are generally based on fitted parameter estimates, so uncertainty in those estimates may be absent unless deliberately incorporated. They also depend on the adequacy of the assumed random-effects and residual distributions. Predictive checks are model diagnostics, not automatic pass/fail tests, and a check of one summary cannot establish adequacy for every scientific use.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A check is intended to evaluate trough concentrations for new individuals under the same repeated-dose schedule. Should individual effects be redrawn or retained?
- A conditional check holds each estimated individual effect fixed and generates new residual errors. Which source of variability is not newly represented?
- In a hypothetical check, 5 of 20 observed records exceed a threshold, while the conditional replicate median is 2 of 20. Calculate both proportions and state what the comparison suggests.
- Why is it misleading to compare the observed population standard deviation directly with a conditional residual standard deviation?
- A population-level check shows excess low troughs, while a conditional check does not. Give one plausible interpretation and one reason not to treat it as a unique diagnosis.
Solutions. (1) Redraw effects from the fitted population distribution, because the target is new individuals; retain the observed design. (2) Between-individual variation from new random-effects draws is not newly represented. (3) The observed proportion is \(5/20=0.25\), or 25%; the replicate median is \(2/20=0.10\), or 10%. The observed threshold-exceedance rate is higher than the replicate median, suggesting a discrepancy for this conditional target, but the median alone does not describe the full replicate range. (4) The population standard deviation includes between-individual and residual variation, whereas a conditional residual standard deviation describes variation around a fixed individual’s conditional mean. (5) The population random-effects distribution or its relation to covariates could be implicated because the population check redraws effects. It is not a unique diagnosis because structural, residual, or other model features can also affect low trough predictions.