Conditional and Population-Level Residuals

Why Two Prediction Scales Matter

A PD model can describe a population adequately while fitting a particular subject imperfectly, or it can fit individual observations after subject-specific adjustments while still missing a feature shared across the population. Residuals can help separate these questions, but only if we keep track of which prediction was used to calculate them.

A conditional prediction uses the model’s estimated subject-specific random effects. It asks how close the observation is to the fitted trajectory for that subject. A population prediction does not use that subject’s estimated random effects. It asks how the observation compares with the model’s prediction at the population level. Conditional residuals therefore emphasize individual fit; population-level residuals retain variation between subjects and can reveal patterns that individual adjustment partly conceals.

The typical-subject prediction sets random effects to zero. The marginal population mean averages predictions over the random-effects distribution. These are equal in a linear model where the random effects enter additively with mean zero, but need not be equal in a nonlinear model. In what follows, “population prediction” refers to the marginal mean when that is available; if a diagnostic uses \(f_{ij}(0)\), it should be labelled as a typical-subject comparison.

Model and Residual Scales

For a continuous response with additive, homoscedastic Gaussian residual error, suppose

Here \(\sigma^2\) is the conditional residual-error variance and \(\Omega\) is the between-subject random-effects covariance matrix. This formulation assumes the conditional mean function and random-effects distribution are specified, residual errors have mean zero and constant variance \(\sigma^2\), and records are conditionally independent given the random effects. These assumptions define the simple scales used here; a changing residual variance requires a different denominator and is addressed separately in the course.

The conditional residual is naturally standardized by the conditional residual standard deviation \(\sigma\). The population-level residual includes both residual error and variation due to random effects. For a scalar observation, its marginal predictive variance is

This is the law of total variance: average the within-subject variance, then add the variance of the conditional mean across subjects. A marginally standardized population residual divides the difference from the marginal mean by the square root of this variance. It is a scale adjustment, not a guarantee that residuals are independent standard normal values, especially in nonlinear models or when parameters have been estimated from the same data.

Worked Example: A Subject Offset Hidden by Conditional Predictions

Why the Two Standardizations Differ

Under the additive random-intercept model in the example, write \(Y_{ij}=\mu_{ij}+\eta_i+\epsilon_{ij}\), where \(\eta_i\) has variance \(\omega^2\) and \(\epsilon_{ij}\) has variance \(\sigma^2\). The conditional residual varies on the \(\sigma\) scale, because the subject effect is included in the conditional prediction. The population residual is \(Y_{ij}-\mu_{ij}=\eta_i+\epsilon_{ij}\), so its variance is \(\omega^2+\sigma^2\). This derivation explains why dividing both residual types by \(\sigma\) would make population residuals look artificially large whenever between-subject variation is substantial.

For a general nonlinear model, the variance of \(f_{ij}(\eta_i)\) may depend on time, dose, covariates, and the random-effects distribution. It should then be calculated under that model, often by numerical integration or simulation, rather than by assuming one constant population standard deviation. The typical-subject prediction \(f_{ij}(0)\) may also differ from the marginal mean. For example, curvature in \(f\) can make the average of subject predictions differ from the prediction at the average random effect.

Reading Patterns and Checking Model Fit

Use the two residual types together. A pattern present in conditional residuals can indicate that the fitted individual trajectories miss a feature that subject-specific random effects do not explain. A pattern in population-level residuals that largely disappears conditionally may instead reflect systematic subject offsets or an inadequate description of between-subject variability. Patterns that remain in both views deserve particular attention: they may indicate a mismatch in the population mean structure, the response-time course, or the covariate specification.

A useful diagnostic sequence is to inspect the prediction definition, plot residuals against relevant design variables, and compare patterns across subjects. Conditional residual plots are useful for within-subject deviations after individual adjustment. Population-level residual plots retain between-subject differences and are useful for assessing how observations compare with population predictions. For repeated observations, population residuals from the same subject are generally correlated because they share random effects; treating every record as independent can exaggerate the apparent amount of information.

These residuals complement rather than replace other checks. Prediction plots and simulation-based checks assess aspects of model performance that a residual plot alone may miss. Conditional predictions can look reassuring when individual estimates are flexible or strongly informed by the same observations being checked. Population predictions avoid that particular subject-specific adjustment but include genuine between-subject variability, so residuals may be broad even when the population model is reasonable.

Common Mistakes and Limitations

• Comparing raw magnitudes as if the scales were identical. Conditional residuals reflect residual error after accounting for a subject effect; population residuals also reflect between-subject variation. Use the appropriate predictive standard deviation, and retain units when reporting raw residuals.
• Calling every large population residual a population-model failure. An observation can be far from the typical prediction because its subject has an ordinary random effect. Check the marginal predictive scale and compare with conditional residuals.
• Treating a small conditional residual as proof of model adequacy. Subject-specific estimates can absorb some discrepancies, and using the same data to estimate those effects can make conditional fit look better. Check population predictions and patterns across subjects as well.
• Equating \(f(0)\) with the marginal mean in every model. This equality holds for linear additive random effects with mean zero, not generally for nonlinear functions. State which population prediction is used.
• Assuming marginal residuals are independent. Repeated records from one subject share random effects, so their population-level residuals can be correlated.

Residuals are diagnostic summaries, not definitive tests. Their interpretation depends on the model, the prediction target, the data used to estimate individual effects, and the assumed error structure. A convincing assessment combines conditional and population-level views with other checks and considers whether a pattern recurs across subjects.