Normalized Prediction Distribution Errors

1. What Are Normalized Prediction Distribution Errors?

Normalized Prediction Distribution Errors (NPDE) are simulation-based diagnostics used to evaluate whether observed data are consistent with the predictive distribution generated by a population PK model.

The central idea is different from simply asking whether an observed concentration is close to a model-predicted concentration. Instead, NPDE asks where the observation falls within the distribution of concentrations that the model predicts for comparable observations.

NPDE are particularly useful in population PK because the model contains multiple sources of variability, including between-subject variability and residual unexplained variability. Simulation allows those sources of variability to contribute to the reference distribution used for the diagnostic.

2. Why Use NPDE Instead of Ordinary Residuals?

Traditional residual diagnostics compare observations with a model prediction, but the meaning of the residual depends on the scale of the data and on the prediction being used.

For example, an observed concentration can be compared with an individual prediction using a prediction error such as:

However, this difference does not directly answer whether the observation is unusual relative to the full predictive distribution implied by the population model.

NPDE therefore provide a complementary diagnostic to standard residual plots and visual predictive checks. They should not be viewed as a replacement for the broader model-diagnostic framework.

3. The Simulation Behind NPDE

NPDE require repeated simulation from the fitted population PK model. The simulations should reproduce the relevant study design, including dosing, sampling times, covariates, variability, and residual-error assumptions.

If \(M\) replicate datasets are simulated, each observed concentration \(y_i\) has a corresponding collection of simulated concentrations:

The position of the observed value within this simulated distribution is then used to construct a probability-based diagnostic.

4. How Are NPDE Calculated?

The calculation can be understood as a sequence of transformations.

1. Fit the population PK model. Estimate the structural parameters, between-subject variability, residual-error model, and relevant covariate effects.
2. Simulate replicate datasets. Generate many datasets from the fitted model while preserving the relevant design and variability structure.
3. Construct the predictive distribution. At each observation, collect the corresponding simulated values.
4. Determine the percentile position of the observation. Estimate where the observed value falls within its simulated predictive distribution.
5. Transform the percentile to a normal scale. Apply the inverse standard-normal cumulative distribution function.
6. Assess the resulting NPDE distribution. Under an adequate model, NPDE should approximately follow \(N(0,1)\), subject to simulation and finite-sample variation.

Let \(F_i(y_i)\) represent the model-based predictive cumulative distribution evaluated at the observed value \(y_i\). The probability transform is:

The normalized prediction distribution error is then obtained by applying the inverse standard-normal transformation:

where \(\Phi^{-1}\) is the inverse cumulative distribution function of a standard normal distribution.

5. Why Correlation Between Observations Matters

A population PK dataset usually contains multiple observations from the same individual. Those observations are not necessarily independent because they share subject-specific random effects.

For example, two concentrations from the same individual can both be influenced by that individual's clearance and volume of distribution.

Consequently, a rigorous NPDE calculation accounts for the covariance structure among observations rather than treating every concentration as an isolated independent measurement.

In matrix notation, the predictive distribution for a subject can be represented conceptually as a multivariate distribution:

where \(\boldsymbol{\eta}_i\) represents subject-level random effects and \(\boldsymbol{\Sigma}\) represents relevant residual-error structure.

This distinction is important because an observation can have a reasonable marginal percentile while the joint pattern of several observations is poorly represented by the model.

6. What Should NPDE Look Like?

If the model adequately describes the observed data-generating process, the NPDE distribution should be approximately:

This gives several useful diagnostic expectations.

These are diagnostic expectations, not rigid pass/fail rules. Simulation size, sampling design, parameter uncertainty, and the number of observations all affect how closely an empirical NPDE distribution resembles the theoretical standard normal distribution.

7. How Are NPDE Visualized?

NPDE are most informative when examined using several complementary plots.

7.1 Histogram or density plot

A histogram of NPDE can be compared with the density of a standard normal distribution. A well-behaved diagnostic should be broadly centered near zero and have a dispersion close to one.

7.2 Quantile-quantile plot

A normal Q-Q plot compares empirical NPDE quantiles with theoretical standard-normal quantiles. Systematic curvature can reveal departures from normality that are less obvious in a histogram.

7.3 NPDE versus time

Plotting NPDE against time can reveal temporal patterns. For example, a model may perform adequately during the elimination phase but systematically misrepresent concentrations during absorption or distribution.

7.4 NPDE versus predictions

Plotting NPDE against predictions can reveal trends suggesting that the residual or structural model behaves differently at low and high concentrations.

7.5 NPDE versus covariates

Relationships with body weight, renal function, age, dose, or other covariates can indicate that an important systematic relationship has not been adequately represented.

8. Worked Example: Interpreting an NPDE

Suppose a population PK model is fitted to concentration data from 100 subjects. The fitted model is then used to generate 1,000 replicate datasets under the same dosing and sampling design.

For one observed concentration, suppose the observed value lies at approximately the 84th percentile of its model-based predictive distribution.

Step 1: Convert the percentile to a probability

Step 2: Transform to the standard-normal scale

This observation therefore has an NPDE of approximately \(+1\). It is above the center of the model's predictive distribution, but it is not by itself evidence of substantial model inadequacy.

Step 3: Consider the full collection of observations

Suppose the complete dataset produces NPDE values with an empirical mean of approximately 0.02 and variance of approximately 1.04, with no obvious trend against time or important covariates.

Those results would be broadly consistent with the expected behavior of NPDE under an adequate model.

Now consider a different model whose NPDE have a mean of 0.65 and show a clear increasing trend with time. That pattern would suggest that the observed data tend to occupy systematically higher positions in the model's predictive distributions and that the discrepancy is not uniform over time.

The next step would be to investigate the model components responsible for that pattern rather than simply labeling the model as inadequate.

9. NPDE vs. Other Population PK Diagnostics

No single diagnostic establishes model adequacy. NPDE are best interpreted alongside structural-model diagnostics, parameter estimates, goodness-of-fit plots, VPCs or prediction-corrected VPCs, and scientific plausibility.

10. What NPDE Do Not Tell You Automatically

NPDE are powerful diagnostics, but they do not eliminate the need for careful model evaluation.

• NPDE depend on the fitted model. They assess the consistency of the data with the model that generated the simulations.
• Simulation quality matters. An insufficient number of simulations can make the estimated predictive distributions noisy.
• Model assumptions matter. Incorrect residual-error, random-effects, or covariate assumptions can influence the diagnostic.
• Sampling design matters. Sparse sampling can limit the ability to detect certain types of model misspecification.
• A roughly normal NPDE distribution does not prove the model is biologically correct. Different models can sometimes produce similar diagnostics over the observed range.
• Overall agreement can hide local problems. A reasonable overall NPDE distribution can coexist with important deviations at particular times or covariate values.

11. A Practical NPDE Workflow

1. Fit the population PK model. Establish the structural model, variability model, covariate model, and residual-error model.
2. Check the basic diagnostics first. Review parameter estimates, goodness-of-fit plots, residuals, and plausibility.
3. Define the simulation design. Reproduce the relevant dosing, sampling, covariates, and variability structure.
4. Generate many replicate datasets. The number of simulations should be sufficient for stable estimation of the predictive distributions.
5. Calculate NPDE. Locate each observation within its model-based predictive distribution and transform its probability to the normal scale.
6. Inspect the NPDE distribution. Review the mean, variance, histogram, and Q-Q plot.
7. Inspect NPDE against explanatory variables. Plot NPDE versus time, predictions, dose, and important covariates.
8. Investigate systematic patterns. Determine whether discrepancies suggest structural, covariate, variability, or residual-error misspecification.
9. Integrate the evidence. Interpret NPDE together with other model diagnostics rather than in isolation.

12. Common NPDE Patterns and What They May Suggest

These interpretations are starting points for investigation rather than automatic diagnoses. The same visual pattern can sometimes arise from different aspects of a population PK model.

13. How NPDE Complements a VPC

A visual predictive check asks whether the observed distribution of concentrations over time is compatible with distributions generated by the model. NPDE approaches the same general question through a probability transformation that produces a standardized diagnostic for individual observations.

The two diagnostics therefore provide different views of simulation-based model evaluation.

A VPC may reveal that the model misses a particular percentile at a particular time, while an NPDE-versus-time plot can provide a complementary indication that observations are systematically displaced within their predictive distributions during that period.

Using both can make it easier to distinguish a global distributional problem from a localized time-dependent discrepancy.

14. What Does a "Good" NPDE Result Mean?

A well-behaved NPDE diagnostic means that the observed concentrations are broadly consistent with the predictive distributions generated by the fitted model.

In practical terms, one would generally look for:

• NPDE centered approximately around zero.
• NPDE variance approximately equal to one.
• A distribution reasonably consistent with normality.
• No major systematic trend with time.
• No important unexplained relationship with key covariates.
• Consistency with the other model diagnostics.

This is evidence of predictive adequacy relative to the evaluated data and model assumptions. It is not proof that every component of the model is uniquely correct.

15. The NPDE Equations in One Place

The conceptual calculation can be summarized in three steps.

Step 1: Predictive cumulative probability

where \(F_i\) is the predictive cumulative distribution for observation \(i\).

Step 2: Probability transformation

When the predictive distribution is correctly specified, the probability-transformed observations should be approximately uniform.

Step 3: Normalization

Therefore, under the model assumptions, the resulting NPDE should be approximately standard normal:

For repeated observations within subjects, the full calculation must account for the dependence structure rather than simply applying an independent marginal transformation to every observation.

Where to Go Next

A natural progression is to study Visual Predictive Checks for Population PK and Prediction-Corrected Visual Predictive Checks, which provide complementary simulation-based views of model adequacy.

From there, useful related topics include bootstrap validation, external validation, eta shrinkage, epsilon shrinkage, and simulation-based qualification of population PK models.