NPDE with Sparse Within-Subject Sampling

Why Sparse Sampling Changes What NPDE Can Tell You

A subject with only one or two observations contributes little direct evidence about how that subject's outcomes co-vary over time. This matters when interpreting normalized prediction distribution errors (NPDE): whitening uses a model-implied covariance across a subject's observation times, but sparse observed records may be unable to show whether that covariance adequately describes real within-subject dependence.

The distinction is important. Simulations from a fitted model can produce a numerical covariance matrix at the observed times, even when the data are too sparse to assess the covariance structure reliably. A successful matrix calculation is not evidence that the model's temporal dependence is correct. With very sparse schedules, marginal predictive checks—whether individual observations agree with their predictive distributions—may be more informative than checks that depend on within-subject relationships.

Definition: For subject \(i\), let \(m_i\) be the number of observed outcomes, and let \(Y_i\) be the vector of those outcomes in observation-time order. The within-subject predictive covariance is the covariance matrix of the model's predictive distribution for \(Y_i\) at that subject's observation times. A sparse within-subject schedule has small \(m_i\), so the observed record contains few outcome pairs with which to assess dependence.

What Whitening Assumes—and What It Establishes

NPDE first places observed outcomes on a normal-score scale using their predictive distributions. In the usual simulation-based calculation, simulated outcomes at each observation time are converted to marginal normal scores, and their covariance matrix is estimated. If \(Z_i\) is the vector of observed normal scores, let \(\mu_i\) and \(C_i\) denote the simulated mean vector and covariance matrix on that scale. If \(C_i\) is positive definite, write its Cholesky factorization as \(C_i=L_iL_i^\mathsf{T}\). The whitened vector is

D_i=L_i^{-1}(Z_i-\mu_i).

Under a correctly specified multivariate normal predictive distribution, with the correct observation schedule and a correctly calculated covariance, \(D_i\) has mean zero and identity covariance. Its components are then independent standard normal variables. For non-normal joint predictive distributions, covariance whitening ensures zero covariance under the stated transformation, but zero covariance alone does not guarantee independence. Thus, normal-looking NPDE marginals and small sample correlations do not prove that every aspect of the joint predictive distribution is correct.

Sparse sampling affects this assessment in two different ways. First, it reduces the number of observed within-subject pairs available to diagnose dependence. Second, it may leave some covariance features unidentified from the data: if nearly all subjects contribute only one measurement, the observed records contain no within-subject pairs at all. In contrast, the simulation-based \(C_i\) is generated from the fitted model and the subject's observed times. It is a model-implied quantity, not an empirical confirmation of that model's covariance.

A Short Justification for the Whitening Step

The algebra explains both the purpose and the limits of whitening. Suppose, for this justification, that \(Z_i\) is multivariate normal with mean \(\mu_i\) and positive-definite covariance \(C_i=L_iL_i^\mathsf{T}\). A linear transformation changes a covariance matrix according to \(\operatorname{Cov}(AZ)=A\operatorname{Cov}(Z)A^\mathsf{T}\). Using \(A=L_i^{-1}\),

\begin{aligned} \operatorname{Cov}(D_i) &=L_i^{-1}C_i(L_i^{-1})^\mathsf{T}\\ &=L_i^{-1}L_iL_i^\mathsf{T}(L_i^{-1})^\mathsf{T}\\ &=I. \end{aligned}

Also, \(E(D_i)=L_i^{-1}(E(Z_i)-\mu_i)=0\). A multivariate normal vector with zero mean and identity covariance has independent standard normal components. This last conclusion uses the multivariate normal assumption; for a general distribution, the calculation establishes zero covariance, not independence.

When \(m_i\) is small, this transformation may be computationally straightforward because the matrix is small. But its interpretation remains limited by the number and arrangement of observations in the data. A well-conditioned \(2\)-by-\(2\) covariance matrix computed from simulations does not create many observed pairs with which to assess whether the assumed correlation is appropriate.

Worked Example: Whitening Two Sparse Observations

Consider one hypothetical subject with two observations. On the normal-score scale, suppose the predictive covariance from simulations is

C_i= \begin{pmatrix} 1&0.8\\ 0.8&1 \end{pmatrix}, \qquad Z_i-\mu_i= \begin{pmatrix} 1.0\\ 1.1 \end{pmatrix}.

The covariance is positive definite because its eigenvalues are \(1.8\) and \(0.2\), both positive. Its Cholesky factor is

L_i= \begin{pmatrix} 1&0\\ 0.8&0.6 \end{pmatrix}, \qquad L_iL_i^\mathsf{T}= \begin{pmatrix} 1&0.8\\ 0.8&1 \end{pmatrix}.

Solve \(L_iD_i=Z_i-\mu_i\). The first component is \(D_{i1}=1.0\). The second satisfies \(0.8D_{i1}+0.6D_{i2}=1.1\), so

D_{i2}=\frac{1.1-0.8(1.0)}{0.6} =\frac{0.3}{0.6} =0.5.

Thus the whitened vector is \((1.0,0.5)^\mathsf{T}\). The second normal score is not treated as an independent deviation of \(1.1\); whitening accounts for the model-implied correlation with the first. This calculation shows how the transformation works, not whether the assumed correlation of \(0.8\) is supported by sparse observed records.

How Few Pairs Affect Dependence Assessment

A sample correlation calculated from observed residuals or NPDE components requires paired values from subjects. With sparse schedules, the relevant sample size is often the number of subjects contributing comparable pairs—not the total number of observations. For example, 100 subjects with one observation each provide 100 outcomes but zero within-subject pairs. If only 10 subjects have two suitably comparable observations, there are at most 10 such pairs for that particular comparison.

Pooling pairs across subjects can be useful only when those pairs represent a common dependence feature. Differences in observation times, doses, covariates, or predictive scales can make a pooled correlation hard to interpret. Conversely, fitting or diagnosing a distinct covariance pattern for every subject is not supported merely because each subject has a simulated covariance matrix. Simulation supplies the model's prediction; observed data still determine how much the model can be challenged.

Worked Example: A Noisy Correlation From Five Subjects

Suppose five hypothetical subjects each have two comparable normal-score residuals, \(x\) and \(y\). The paired values are \((-2,-1),(-1,1),(0,-1),(1,1),(2,0)\). Both sample means are zero. The sums needed for the sample correlation are

\sum x_iy_i=2-1+0+1+0=2,\quad \sum x_i^2=10,\quad \sum y_i^2=1+1+1+1+0=4.

Therefore,

r=\frac{\sum x_iy_i} {\sqrt{\left(\sum x_i^2\right)\left(\sum y_i^2\right)}} =\frac{2}{\sqrt{10(4)}}=0.3162.

This value is a weak-to-moderate positive sample correlation, but five pairs provide little basis for a precise conclusion about the population dependence. In particular, \(r=0.3162\) is not evidence that the true correlation is exactly \(0.3162\), nor does it establish that the model's covariance is correct. The example illustrates why a dependence diagnostic can be inconclusive even when its arithmetic is valid.

When Marginal Predictive Checks Are More Informative

A marginal predictive check compares observed outcomes with their predictive distributions one observation at a time, without requiring the observed within-subject covariance to be well informed. For example, one can assess whether observed values fall within predictive intervals at the corresponding observation times, or compare observed and simulated marginal distributions. Such checks still require appropriate simulations and meaningful comparison groups. They do not test whether repeated outcomes within the same subject have the right dependence.

With sparse schedules, marginal checks can be more reliable in a relative sense: they use the available individual observations to assess location, spread, and distributional calibration, while a within-subject dependence check may rest on very few pairs. “More reliable” does not mean conclusive. A small sample gives uncertain coverage summaries, and a good marginal fit can coexist with incorrect covariance, serial dependence, or joint tail behavior.

Worked Example: Marginal Coverage With One Observation Per Subject

Suppose 20 hypothetical subjects each contribute one concentration measurement at a comparable nominal time. For each subject, model simulations give a 90% predictive interval at that time. Eighteen observed concentrations fall inside their respective intervals. The observed coverage proportion is

\hat{p}=\frac{18}{20}=0.90.

Under independent subjects and correctly calibrated 90% intervals, the approximate standard error of this coverage proportion is

SE(\hat{p})= \sqrt{\frac{0.90(1-0.90)}{20}} =\sqrt{0.0045} =0.0671.

The observed 90% coverage is consistent with the nominal level, but its standard error is about 6.7 percentage points, so this small check is not precise. More importantly, with only one observation per subject there are no within-subject pairs: neither the observed data nor this coverage result can assess within-subject correlation. The marginal check answers a question the design can address; it does not answer the dependence question.

Diagnostics, Comparisons, and Limitations

Interpret NPDE diagnostics in light of the schedule and the covariance calculation. Warning signs include a covariance matrix that is singular or nearly singular, a Cholesky factorization that fails, or NPDE results that change markedly under reasonable changes to simulation count or numerical settings. A very small eigenvalue or large condition number indicates that some linear combinations of scores have little simulated variability, making whitening sensitive to small numerical errors. Check that the covariance is calculated for the actual observation times and that the same score transformation is applied to observed and simulated outcomes.

Also examine what the data can support. Tabulate how many subjects contribute each type of within-subject pair, and avoid treating a large total number of records as if it were a large number of independent pairs. A dependence test with few pairs may have low power: failure to detect dependence is not proof of independence. If marginal NPDE calibration looks acceptable but dependence diagnostics are unstable or weakly informed, report that distinction rather than declaring the joint predictive model validated.

Approach	What it assesses	Main limitation under sparse schedules
Marginal predictive checks	Whether individual outcomes agree with their predictive distributions	Do not assess within-subject dependence; limited sample sizes still produce uncertainty
NPDE with whitening	Marginal normal-score behavior after accounting for model-implied covariance	Dependence assessment relies on few observed pairs and on a credible covariance calculation
Visual predictive checks	Agreement between observed summaries and simulated predictive summaries over the chosen display scale	Summaries can conceal dependence and may be noisy when data are sparse

These approaches answer related but different questions. NPDE whitening makes a dependence adjustment using the fitted model's covariance; it does not estimate that covariance from each subject's few observations. A visual or marginal check can reveal distributional mismatches that an NPDE summary misses, but it cannot substitute for adequate repeated measurements when the scientific question concerns within-subject dependence. The appropriate conclusion may therefore be that marginal prediction is assessed more credibly than covariance structure.

Common Mistakes

Confusing a computable covariance with an empirically validated covariance. Simulations can estimate the model-implied covariance accurately, but sparse observed records may still provide little evidence that the model has the right dependence.
Reading a non-significant dependence diagnostic as proof of independence. Few pairs can make a test insensitive. State that the diagnostic is inconclusive when information is limited.
Using the total number of observations as the dependence sample size. Dependence requires paired observations from subjects, so count informative pairs and subjects, not just records.
Claiming that marginal agreement validates the joint distribution. Marginal checks assess individual predictive distributions; they do not establish correct within-subject covariance or independence.
Assuming whitening guarantees independence in every model. Identity covariance implies independence for a multivariate normal vector, but not for every non-normal joint distribution.

Key takeaway: Sparse sampling can make NPDE whitening numerically possible while leaving within-subject dependence weakly assessed. Report what the marginal checks support, describe the limited information in available pairs, and avoid interpreting an absence of detected dependence as proof that the covariance model is correct.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Twenty subjects each contribute one observation. How many within-subject pairs are available to assess correlation?
Why does a positive-definite simulated covariance matrix not prove that the model's within-subject dependence is correct?
Under what assumption does identity covariance of whitened NPDE components imply their independence?
What can a marginal 90% predictive-interval coverage check assess, and what can it not assess?

Solutions. (1) Zero: no subject contributes two observations. (2) The matrix describes dependence implied by the fitted model and simulations; the observed schedule may contain too few pairs to validate that dependence. (3) The whitened vector must be multivariate normal; without that assumption, identity covariance establishes uncorrelated components but not necessarily independent components. (4) It can assess whether individual observed outcomes fall within their marginal predictive intervals at approximately the expected rate, subject to sampling uncertainty. It cannot assess within-subject dependence when each subject has only one observation.