Why Check NPDE Dependence?
NPDE diagnostics ask more than whether residuals are centered near zero and have a roughly standard normal distribution. For repeated observations, a model might reproduce the marginal distribution at each observation while leaving a pattern across observations within subjects. For example, successive NPDE may tend to have the same sign, or may rise and fall systematically with time. Such structure suggests that the joint predictive behavior has not been captured adequately, even if a histogram looks plausible.
The distinction matters because standard NPDE are constructed by covariance whitening and marginal normal-score transformations. Whitening makes the transformed predictive deviations uncorrelated when the covariance is correctly specified and nonsingular. It does not, in general, make them independent. Marginal normal-score transformation targets standard normal margins, but it does not make a nonnormal joint distribution independent either. Thus the purpose of this tutorial is to assess observable dependence patterns without making a stronger claim than the transformation warrants.
What the NPDE Transformation Guarantees
This definition separates two operations. First, whitening uses the predictive covariance across the observations in a subject. When the covariance is correct, it gives \(\operatorname{Cov}(Z_i)=I\), where \(I\) is the identity matrix. Second, the marginal transformation maps each continuous predictive margin to a standard normal distribution. These statements concern covariance and marginal distributions; neither alone is a statement of mutual independence.
There is an important special case. If the predictive vector \(Y_i\) is jointly multivariate normal with mean \(\mu_i\) and covariance \(\Sigma_i\), then \(Z_i\) is jointly multivariate standard normal. Its identity covariance matrix implies that its coordinates are independent standard normal variables. The marginal normal-score transformation leaves those coordinates unchanged in distribution. In that special case, independence is the appropriate target, provided the joint predictive model is correct.
For a general nonnormal predictive distribution, the NPDE margins are intended to be standard normal, but dependence can remain after the marginal transformations. In addition, although the whitened coordinates have zero pairwise covariance, a nonlinear marginal transformation can change their correlations. Therefore, inspect dependence in the final NPDE themselves; do not infer their independence solely from the whitening step. A full joint transform, such as a correctly specified Rosenblatt transform for a continuous joint distribution, has different properties, but it is not the standard marginal NPDE construction.
Subject-Level Displays and Summaries
Keep each subject's observations together and order them by the relevant design variable, usually time. A plot of NPDE against time can reveal trends, runs of positive or negative values, cycles, or changes in spread. Connecting observations within a subject can make sequences visible, but use a clear subject-level display: if many subjects are overplotted, the individual paths can become unreadable. Faceting a subset of subjects or plotting several thin paths with modest transparency can help.
Numerical summaries should respect that observations from the same subject are not independent sampling units. A useful descriptive summary is the lag-one correlation: for each subject, pair each NPDE with the next NPDE in time and summarize the association across those pairs. The precise pairing rule should be specified in advance. With irregular observation times, “next observation” can represent very different elapsed times; group pairs by a stated time-lag range or examine the NPDE against elapsed time rather than treating all adjacent pairs as equivalent.
An overall correlation calculated by pooling every pair can be dominated by subjects with many observations. It also does not, by itself, provide a valid standard error or test if the pooled pairs overlap within subjects. A subject-level display and subject-aware summary are usually more informative than a single pooled number. If formal uncertainty is needed, the resampling or inferential procedure must preserve subjects as clusters; the effective independent sampling units are subjects, not all recorded observations.
Worked Example: Calculating a Lag-One NPDE Correlation
Suppose a hypothetical predictive check gives one pair of consecutive NPDE for each of six independent subjects. Each subject has only this one pair, so the six pairs are not overlapping within subjects. The values are \((-1.5,-1.2)\), \((-0.5,-0.4)\), \((0.5,0.3)\), \((1.5,1.4)\), \((-1.0,0.8)\), and \((1.0,-0.9)\). Let the first coordinate be the earlier NPDE \(x\) and the second the next NPDE \(y\).
The sample means are both zero. The sums of squares and cross-products are \(\sum x^2=7.00\), \(\sum y^2=5.10\), and \(\sum xy=2.55\). For example, the cross-product sum is \(1.80+0.20+0.15+2.10-0.80-0.90=2.55\). The sample correlation is therefore \(r=\frac{\sum(x-\bar{x})(y-\bar{y})}{\sqrt{\sum(x-\bar{x})^2\sum(y-\bar{y})^2}}=\frac{2.55}{\sqrt{7.00(5.10)}}=0.427\).
This hypothetical sample has a positive lag-one association: larger earlier NPDE tend to accompany larger next NPDE. With only six subjects, the estimate is highly uncertain and should be treated as a descriptive signal, not decisive evidence of model failure. The next steps are to inspect the six subject pairs and their time ordering, check the marginal NPDE diagnostics, and assess whether the apparent pattern is stable in a larger set of subjects.
Separating Serial Structure from Marginal Miscalibration
Marginal calibration and dependence are different questions. A marginal histogram or normal quantile plot addresses whether NPDE values, considered one coordinate at a time, resemble a standard normal distribution. A subject-level sequence plot or lag summary addresses whether values from the same subject are related across observations. Neither diagnostic substitutes for the other.
A residual sequence that remains positive for a while and then negative may suggest serial structure, even if the pooled marginal distribution is centered and has a plausible spread. Conversely, a marginal shift or heavy tails can occur without a clear time-ordered pattern. When interpreting either display, consider the observation design, the fitted predictive distribution, and the way the NPDE were estimated. A striking pattern is a diagnostic clue that should prompt investigation, not a standalone proof of a particular model defect.
Worked Example: A Pattern Hidden by a Pooled View
Consider four hypothetical subjects, each with four NPDE in time order: Subject A: \((-1.2,-0.4,0.4,1.2)\); Subject B: \((-1.0,-0.3,0.3,1.0)\); Subject C: \((1.2,0.4,-0.4,-1.2)\); Subject D: \((1.0,0.3,-0.3,-1.0)\).
Across all 16 values, the sum is zero: each increasing sequence is offset by a decreasing sequence. The pooled values are also symmetric about zero. A pooled display can therefore conceal the fact that every subject has a strongly ordered trajectory: A and B increase, while C and D decrease. A plot connecting each subject's points makes these trends apparent immediately.
This example does not show a common positive lag-one association across subjects; the directions oppose one another. It shows why a single pooled marginal view is inadequate for diagnosing within-subject structure. The appropriate interpretation is that there is marked subject-specific time ordering in this small hypothetical set, not that one particular shared serial-correlation parameter has been demonstrated. Further assessment would need more subjects and consideration of the design and predictive model.
Why Zero Correlation Does Not Establish Independence
The limitation of covariance whitening follows directly from the definition of covariance. Zero covariance rules out linear association, but it does not rule out nonlinear dependence. The following construction gives a compact justification. Let \(Z\sim N(0,1)\), and let \(R\) independently take values \(-1\) and \(1\), each with probability \(1/2\). Define \(X=Z\) and \(Y=RZ\).
Both \(X\) and \(Y\) have standard normal margins. Yet \(|X|=|Y|\) with probability one, so they are dependent: knowing \(|X|\) determines \(|Y|\). The covariance matrix is already the identity, so whitening changes nothing. Applying marginal normal-score transformations also changes neither standard normal margin. Thus the resulting coordinates remain dependent despite having standard normal margins and zero covariance. This demonstrates why standard NPDE independence should not be claimed for a general nonnormal joint predictive distribution.
Worked Example: Interpreting Zero Covariance Correctly
Use the construction above. Since \(R\) is independent of \(Z\), \(E(R)=0\), and \(E(Z^2)=1\), the covariance is \(E(XY)=E(RZ^2)=0\). Both coordinates have variance one, so their covariance matrix is the identity. However, the event \(\{|X|>1\}\) is exactly the event \(\{|Y|>1\}\); these events are not independent. For instance, each has probability about \(0.3173\), whereas their intersection also has probability \(0.3173\), not \(0.3173^2\).
The conclusion is not that NPDE are always dependent. Rather, zero covariance and normal margins alone do not establish independence. Independence is justified under a jointly multivariate normal predictive distribution after correct whitening, or by a suitable full joint transform under its own assumptions. For standard NPDE, report what the marginal and dependence diagnostics show, and keep their implications distinct.
Assumptions, Comparisons, and Failure Modes
Interpretation depends on using the predictive distribution appropriate to the check. The simulations and observed records must correspond to the same design and outcome definition; the predictive means, covariances, and marginal distributions must be estimated adequately; and covariance whitening requires a nonsingular covariance matrix. Subjects are ordinarily treated as independent units conditional on the model and design. If a subject has too few observations, there may be little information about that subject's sequence; if the simulation count is limited, estimated marginal transformations can also be noisy.
The main alternatives answer related but distinct questions. Marginal NPDE histograms and normal quantile plots assess marginal calibration, not serial dependence. A VPC compares observed and simulated summaries over time or other design variables, but its aggregate summaries may not reveal individual-level residual sequences. Raw residual plots can show patterns, but their scale and variance may vary with the prediction and observation model. NPDE aim to account for the predictive mean and covariance before assessing standardized deviations, but their dependence interpretation remains conditional on the joint distribution and transformation.
Common warning signs include a systematic rise or fall within many subjects, repeated runs of the same sign, associations concentrated at particular time separations, or a dependence summary that changes substantially when subjects with many observations are excluded. These patterns can reflect omitted time structure, a misspecified covariance model, design-related differences, or simulation and estimation limitations. They are not uniquely diagnostic of any one cause. Check plots and simulation setup before attributing the pattern to a specific model component.
Review the NPDE distribution and relevant normality displays, while remembering that these summarize values one coordinate at a time.
Plot NPDE by time with subject identity visible, and look for trends, runs, or repeated patterns.
Specify the within-subject pairing or time-lag rule, report the resulting descriptive summary, and avoid treating overlapping observations as independent units.
Distinguish evidence about marginal calibration from evidence about serial structure, and do not claim independence solely from identity covariance.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- What does covariance whitening guarantee when the predictive covariance is correctly specified? What does it not guarantee for a nonnormal predictive vector?
- In a hypothetical check, the NPDE histogram is centered and approximately bell-shaped, but many subject-level sequences show consecutive values with the same sign. What distinction does this illustrate?
- Why can a pooled correlation over all consecutive pairs give disproportionate weight to some subjects?
- Under what condition can identity covariance of whitened coordinates justify independence?
- In the example \(X=Z,\;Y=RZ\), why does zero covariance fail to establish independence?
Solutions. (1) Correct whitening gives identity covariance, hence zero pairwise covariance among distinct whitened coordinates; it does not generally establish mutual independence. (2) Marginal calibration can look plausible while serial dependence remains; the marginal distribution and within-subject dependence are separate diagnostics. (3) Subjects with more observations contribute more adjacent pairs, and overlapping pairs within a subject are not independent sampling units. (4) If the predictive vector is jointly multivariate normal, identity covariance implies independent coordinates. (5) The magnitudes are identical, \(|X|=|Y|\), so the variables are dependent despite having zero covariance and standard normal margins.