Interpreting NPDE Distributions

What an NPDE Distribution Can Tell You

Normalized prediction distribution errors (NPDEs) put observed data on a common scale by comparing them with the model’s predictive distribution. Under the reference model and the assumptions used to compute them, NPDEs should follow a standard normal distribution. Interpreting their distribution asks whether the observed discrepancies are centered near zero, have approximately unit spread, and show the shape and tails expected from that reference.

The goal is not to decide that a model is adequate because a summary statistic is close to its target, or inadequate because it is not exactly equal to that target. A finite collection of NPDEs will vary. Interpretation therefore combines several summaries and a normal quantile plot, while asking how much of each apparent departure could plausibly arise from sampling variation.

This reference is conditional on the predictive model being appropriate for the data and design, the predictive simulations representing that model adequately, and the decorrelation being appropriate. Estimated model parameters and a finite number of simulations can also affect the realized NPDE distribution. The finite-sample behavior of NPDEs is therefore relevant when deciding how much weight to give a small departure.

Location and Spread

The NPDE mean summarizes average signed discrepancy on the normalized scale. A mean above zero indicates that NPDEs tend to be positive; a mean below zero indicates that they tend to be negative. If observations are consistently above or below their predictive expectations, this may produce such a shift. However, a positive NPDE is not itself a concentration difference in original units, and the mean alone does not identify a cause. The median is a useful companion: a large difference between mean and median can indicate asymmetry or influential tail values.

The standard deviation describes the spread of NPDEs. The reference value is 1. A substantially smaller spread can indicate that discrepancies are less variable than predicted, while a substantially larger spread can indicate more variability than predicted. These are diagnostic clues, not diagnoses: skewness or a few extreme values can also influence the standard deviation.

For \(n\) approximately independent NPDEs with sample mean \(\bar{x}\) and sample standard deviation \(s\), a rough large-sample standard error for the mean under the standard-normal reference is \(1/\sqrt{n}\). More generally, the estimated standard error of the sample mean is \(s/\sqrt{n}\). This gives a scale for sampling variation, not a guarantee that a departure is unimportant or important. The calculation is most defensible when the NPDEs are sufficiently independent and the mean has a stable sampling distribution.

The justification is direct. If the NPDEs \(X_1,\ldots,X_n\) are independent, each has mean zero and variance one under the reference, and \(\bar{x}=(X_1+\cdots+X_n)/n\), then \(E(\bar{x})=0\) and \(\operatorname{Var}(\bar{x})=1/n\). Taking the square root gives the reference standard error. Replacing the unknown spread by the observed \(s\) gives the estimated standard error above. Dependence, parameter estimation, or other departures from these assumptions can make this approximation inaccurate.

Shape and Tails in a Normal Quantile Plot

A normal quantile plot compares the ordered NPDEs with the quantiles expected from a standard normal distribution. If the plotted points follow an approximately straight line near the identity line, the overall distribution is reasonably compatible with the reference. Systematic curvature is informative: one tail bending away from the line can suggest asymmetry, while both ends departing in opposite directions can suggest tails heavier or lighter than the normal reference. A small number of points at the extremes should be treated cautiously because extreme order statistics are variable in finite samples.

One common plotting convention pairs the \(i\)th ordered NPDE, \(x_{(i)}\), with the theoretical quantile \(z_i=\Phi^{-1}((i-0.5)/n)\), where \(\Phi^{-1}\) is the standard-normal quantile function. The exact plotting-position convention varies slightly, so the important question is whether the pattern departs systematically from the reference, not whether every point lies on a line.

A normal quantile plot is more revealing about shape than a mean and standard deviation alone. Two samples can have similar means and standard deviations but different skewness or tail behavior. Conversely, visible deviations in a plot do not automatically imply model failure: quantile points have sampling variability, especially near the ends. Simulation-based envelopes, generated under the relevant null predictive setup, can show the amount of variation expected in the whole plot. Their construction must reflect the design and NPDE computation being assessed.

Skewness and Tail Summaries

Skewness describes asymmetry. A positive sample skewness indicates a longer or more influential upper side; a negative value indicates a longer or more influential lower side. One descriptive definition is the standardized third central moment, \(g_1=m_3/s^3\), where \(m_3=n^{-1}\sum_{i=1}^n(x_i-\bar{x})^3\). The standard-normal reference has skewness zero. Small nonzero estimates are expected in finite samples, and the estimate can be strongly affected by a few extreme NPDEs.

Tail behavior concerns how often very low or very high values occur relative to the normal reference. The empirical 5th and 95th percentiles can be compared descriptively with the standard-normal values \(-1.645\) and \(1.645\). The excess kurtosis, defined here as \(g_2=m_4/s^4-3\), with \(m_4=n^{-1}\sum_{i=1}^n(x_i-\bar{x})^4\), is another summary of tail weight and peakedness. The normal reference has excess kurtosis zero. Neither a tail quantile nor kurtosis should be interpreted in isolation: the latter is especially sensitive to extreme observations.

Under independent standard-normal sampling, rough large-sample standard errors for sample skewness and excess kurtosis are \(\sqrt{6/n}\) and \(\sqrt{24/n}\), respectively. These approximations give context, not formal decision rules. For example, with \(n=100\), they are about \(0.245\) and \(0.490\). Strong dependence or departures from the reference can invalidate these simple benchmarks; simulation envelopes or other design-appropriate uncertainty assessments are preferable when available.

Reading the Diagnostics Together

Interpretation is strongest when the plot and summaries tell a coherent story. A shifted mean with a similarly shifted median and broadly straight quantile plot suggests a location difference more than a shape difference. A mean near zero with a standard deviation well above one and outward departures at both ends suggests excess dispersion or heavier tails. A mean and standard deviation near their targets do not exclude opposing tail departures or asymmetry.

A useful sequence is to describe the observed pattern first, then qualify it by sampling uncertainty and assumptions:

An overall NPDE distribution compresses observations into a common scale. It can summarize location, spread, and shape, but it does not retain all information about concentration-time behavior. A VPC instead compares observed and simulated data summaries on the measurement scale across time or other groupings. These approaches answer related but different questions; a reassuring overall NPDE distribution does not guarantee that predictions are adequate in every region of the design.

Common Pitfalls and Limitations

• Treating the reference values as sample targets that must be exact. A mean of 0.1 or standard deviation of 1.1 is not automatically concerning; both vary from sample to sample.
• Calling a small p-value or a visible curve a diagnosis. A departure describes incompatibility with a reference distribution, not the reason for it. Formal distribution tests address a separate question.
• Using the mean to summarize the entire distribution. Opposing tail departures can cancel in the mean, and a few extremes can move it.
• Ignoring dependence or computation uncertainty. The simple standard-error calculations rely on approximate independence and can understate uncertainty otherwise.
• Concluding that an overall distribution validates every prediction. Aggregation can hide patterns tied to time, covariates, or particular portions of a concentration profile.

The practical conclusion should therefore match the evidence: state which feature appears shifted or misshapen, describe its magnitude, and indicate whether it is convincing relative to expected sampling variation. Avoid turning a descriptive discrepancy into a causal claim about a particular model component without further investigation.