Deriving the NPDE Covariance Transformation

Why Whiten Predictive Deviations Jointly?

A subject’s repeated observations are often related even after accounting for their different prediction times. For example, a subject-specific random effect can shift several concentrations in the same direction, while residual error contributes variation specific to each measurement. Consequently, a deviation that is surprising at one time may not be independent of a deviation at another time. Scaling each observation by its own standard deviation does not remove this dependence.

The covariance transformation used in NPDE calculations addresses this by treating the observations from one subject as a vector. It uses both the marginal variances and the covariances between observations. The result is a vector of deviations with identity covariance under the predictive model. Under a multivariate normal predictive distribution, those transformed components are also independent standard normal variables.

Here, the predictive distribution is understood to include the sources of variation specified by the predictive check, such as between-subject random effects and residual error. The mean and covariance must correspond to the same conditioning and design as the observations being assessed. For a given subject, the matrix dimension is \(m_i \times m_i\), where \(m_i\) is the number of that subject’s observations.

Deriving the Covariance Transformation

Suppose first that the predictive deviation has mean zero and positive-definite covariance \(\Sigma_i\). A covariance matrix is positive definite when \(a^\mathsf{T}\Sigma_i a>0\) for every nonzero vector \(a\). This condition ensures that all directions in the observation vector have nonzero predictive variance and that the covariance matrix can be inverted.

Because \(\Sigma_i\) is symmetric and positive definite, it has an eigendecomposition \(\Sigma_i=U_i\Lambda_iU_i^\mathsf{T}\), where the columns of \(U_i\) are orthonormal eigenvectors and \(\Lambda_i\) is diagonal with positive eigenvalues. The symmetric inverse square root is \(W_i=U_i\Lambda_i^{-1/2}U_i^\mathsf{T}\). It divides each eigen-direction by the square root of that direction’s predictive variance.

The simplification follows from \(U_i^\mathsf{T}U_i=I\) and from multiplying each positive eigenvalue by its inverse square root on both sides: \(\lambda^{-1/2}\lambda\lambda^{-1/2}=1\). Thus whitening gives zero covariance between distinct transformed components and unit variance for each component. It does not, by itself, establish that the components are normally distributed. If \(q_i\sim N(0,\Sigma_i)\), then the linear transformation gives \(z_i\sim N(0,I)\), whose components are independent standard normal variables.

The symmetric inverse square root is one valid choice, but not the only one. If \(\Sigma_i=L_iL_i^\mathsf{T}\) is a Cholesky factorization, then \(W_i=L_i^{-1}\) also satisfies \(W_i\Sigma_iW_i^\mathsf{T}=I\). These choices can produce different individual coordinates, even though both have identity covariance. The symmetric choice treats directions without a sequential ordering; Cholesky whitening depends on the order of observations.

How Repeated-Measurement Correlation Enters

A simple random-intercept model shows where off-diagonal covariance comes from. Suppose the model prediction for observation \(j\) on subject \(i\) can be written as a fixed mean plus a shared subject effect and an observation-specific residual: \[ Y_{ij}=\mu_{ij}+b_i+\epsilon_{ij}. \] Assume \(E(b_i)=0\), \(\operatorname{Var}(b_i)=\omega^2\), \(E(\epsilon_{ij})=0\), and \(\operatorname{Var}(\epsilon_{ij})=\sigma^2\). Also assume the residuals are independent across observations and independent of \(b_i\). Then each observation has variance \(\omega^2+\sigma^2\), while two different observations on the same subject have covariance \(\omega^2\), because they share \(b_i\).

This covariance is exactly what a joint transformation needs to account for. If the random intercept is ignored, repeated positive deviations may be counted as though they were separate evidence, even though their common direction is expected under the model. Conversely, a deviation pattern in a low-variance contrast direction can be more surprising than marginal scales suggest.

Alternative Whitening and Coordinate Choices

The Cholesky method provides a convenient alternative when a sequential transformation is appropriate. It is useful to see why it can differ from symmetric whitening without contradicting the identity-covariance result.

Assumptions, Diagnostics, and Limitations

A valid covariance whitening step depends on a correctly specified predictive mean and covariance for the subject’s observation design. The deviation should be centered by the predictive mean; otherwise \(E(q_i)\) may not be zero, and whitening will not correct that bias. The covariance matrix must also be symmetric positive definite for an ordinary full-rank inverse square root or Cholesky inverse to exist.

Useful checks include confirming that the covariance is symmetric, inspecting its eigenvalues, and verifying numerically that \(W_i\Sigma_iW_i^\mathsf{T}\) is close to the identity. If a supposedly full-rank covariance has a zero or negative eigenvalue, do not silently treat the ordinary inverse square root as valid. A zero eigenvalue indicates a direction with no predictive variation; a negative eigenvalue is inconsistent with a valid covariance matrix and may indicate numerical or modeling problems. Near-zero eigenvalues can make the transformation highly sensitive to small errors.

After transformation, persistent nonzero means, variances far from one, or dependence in the whitened deviations can indicate a problem with predictive calibration, covariance specification, distributional assumptions, or numerical implementation. These checks are diagnostic rather than a proof that the model is correct. In particular, identity covariance does not guarantee normality. A nonlinear mixed-effects predictive distribution can be nonnormal, even when its mean and covariance are accurately represented. Under nonnormality, whitening removes linear covariance but does not generally make the components independent or normally distributed.

A common mistake is to standardize each time point separately and conclude that the repeated deviations are independent because each has variance one. In the random-intercept example, each marginally standardized deviation still has correlation \(0.6154\). The correction is to transform the vector using its full covariance matrix. A second mistake is to interpret a large whitened contrast as a large raw concentration residual: whitening measures deviation relative to the model’s variance in a direction, so its scale and meaning depend on the covariance structure.

The symmetric inverse square root is especially interpretable in terms of covariance eigen-directions, while Cholesky whitening is computationally convenient and naturally sequential. Neither choice repairs a wrong covariance model, removes nonlinear dependence in general, or substitutes for assessing the predictive distribution. The covariance transformation is one defined step in the NPDE construction, not a standalone guarantee of model adequacy.