When Conditional Weighted Residuals Are Approximately Standard Normal

Why “Approximately Standard Normal” Needs Conditions

A CWRES value is often read against a standard-normal reference: values near zero are common under an adequate model, while large absolute values are less common. That interpretation is not automatic. It depends on how the model is approximated, how individual effects are estimated, and whether the assumed residual variance describes the data. This tutorial explains the conditions behind that reference, rather than treating standard normality as a property guaranteed by the residual’s name.

The earlier material on deriving individual and conditional weighted residuals introduced the residual construction. Here, the central point is that CWRES are approximately standard normal when a local linearization of the individual prediction is adequate and the resulting residual covariance is correctly represented. The exact result is easiest to see in a linear Gaussian model.

Definition: For individual \(i\), let \(y_i\) be the vector of observations, \(f_i(\eta_i)\) the vector of model predictions at individual effect \(\eta_i\), and \(\hat{\eta}_i\) its conditional estimate. The conditional residual vector is \(r_i=y_i-f_i(\hat{\eta}_i)\). A CWRES vector is obtained by standardizing this residual vector using its conditional covariance under a linearized model. If \(S_i=L_iL_i^\mathsf{T}\) is that covariance’s Cholesky factorization, then \(CWRES_i=L_i^{-1}r_i\). The standardized components are approximately standard normal when the model and approximation assumptions are adequate.

The Linear-Gaussian Reference Result

Consider one individual and temporarily suppress the individual subscript. Suppose the observations follow the linear random-effects model

y=f_0+G\eta+\epsilon,\qquad \eta\sim N(0,\Omega),\qquad \epsilon\sim N(0,R),

where \(f_0\) is the population prediction vector, \(G\) is the sensitivity of predictions to the individual effects, \(\Omega\) is their covariance matrix, and \(R\) is the residual-error covariance matrix. Assume \(\eta\) and \(\epsilon\) are independent, and that \(\Omega\) and \(R\) are positive definite. Then \(e=y-f_0\) has covariance \(V=G\Omega G^\mathsf{T}+R\).

In this model, the conditional mean of \(\eta\) given the observations is \(\hat{\eta}=\Omega G^\mathsf{T}V^{-1}e\). The corresponding fitted residual is

r=e-G\hat{\eta} =\left(I-G\Omega G^\mathsf{T}V^{-1}\right)e =RV^{-1}e.

The final equality follows from \(V=G\Omega G^\mathsf{T}+R\): therefore \(I-G\Omega G^\mathsf{T}V^{-1}=I-(V-R)V^{-1}=RV^{-1}\). Since \(e\) has mean zero and covariance \(V\), this residual has mean zero and covariance

S=\operatorname{Var}(r) =RV^{-1}\operatorname{Var}(e)V^{-1}R =RV^{-1}R.

Here \(R\) is symmetric, as a covariance matrix is. If \(S=LL^\mathsf{T}\), then \(z=L^{-1}r\) has mean zero and covariance \(I\). Because \(e\), and hence \(r\), is normally distributed in this reference model, \(z\sim N(0,I)\). Thus each component is standard normal, and the components are independent in this exact multivariate normal case. In practice, nonlinear predictions are locally approximated by a linear model and individual effects are estimated from data. Those steps make the reference approximate rather than exact.

Specify the distribution.
Use a suitable random-effects distribution and residual-error distribution on the scale being analyzed. The derivation above assumes independent normal random effects and normal residual errors.

Linearize the prediction.
Represent the local effect of individual parameters on predictions with a sensitivity matrix \(G\). This is reliable only when curvature over the relevant range of individual effects is not too influential.

Use the conditional covariance.
Standardize the residual vector with its covariance, including the dependence induced by estimating individual effects. Standardizing each residual as though it were unrelated to the others can give a different result.

Interpret as a reference, not a guarantee.
Even if the approximation is reasonable, parameter estimation, numerical error, and model misspecification can affect the observed CWRES distribution.

Worked Calculations

Worked Example: Whitening Two Correlated Conditional Residuals

Suppose a hypothetical individual has two observations from a linearized model. Let the prediction deviations from the population prediction be \(e=(1,-1)^\mathsf{T}\), let \(G=(1,1)^\mathsf{T}\), let \(\Omega=0.25\), and let \(R=I\). These are hypothetical values chosen to illustrate the calculation.

First, \(V=G\Omega G^\mathsf{T}+R\), so

V= \begin{pmatrix}1.25&0.25\\0.25&1.25\end{pmatrix}, \qquad V^{-1}= \begin{pmatrix}5/6&-1/6\\-1/6&5/6\end{pmatrix}.

The conditional estimate is \(\hat{\eta}=\Omega G^\mathsf{T}V^{-1}e=0\), because the two deviations cancel. Hence \(r=e-G\hat{\eta}=(1,-1)^\mathsf{T}\). The residual covariance is

S=RV^{-1}R= \begin{pmatrix}5/6&-1/6\\-1/6&5/6\end{pmatrix}.

A lower-triangular Cholesky factor is \(L\approx\begin{pmatrix}0.9129&0\\-0.1826&0.8944\end{pmatrix}\). Therefore

CWRES=L^{-1}r \approx \begin{pmatrix}1.095\\-0.8944\end{pmatrix}.

The two unwhitened residuals are opposites, but their standardized values are not simply those residuals divided by one common standard deviation: the conditional residuals have nonzero covariance. The calculation accounts for that dependence. Under the stated linear Gaussian model, the transformed vector has the standard-normal reference distribution; one pair of values does not itself establish that the model is adequate.

Worked Example: A Single Observation With Conditional Shrinkage

For a hypothetical single observation, take the linear model \(y=f_0+\eta+\epsilon\), with \(\Omega=0.25\), \(R=0.25\), and observed deviation \(e=y-f_0=0.50\). The random effect and residual error are independent and normally distributed, as required by the reference calculation.

Here \(V=\Omega+R=0.50\), and the conditional estimate is \(\hat{\eta}=\Omega V^{-1}e=(0.25/0.50)(0.50)=0.25\). Thus \(r=e-\hat{\eta}=0.25\). Its variance is \(S=RV^{-1}R=(0.25)(1/0.50)(0.25)=0.125\). The CWRES is

CWRES=\frac{r}{\sqrt{S}} =\frac{0.25}{\sqrt{0.125}} \approx 0.7071.

The individual estimate accounts for half of the observed deviation in this example. The residual is therefore not the original deviation \(0.50\), and its appropriate standard deviation is not obtained by looking only at the residual-error variance. Both the conditional estimate and its covariance correction matter.

Worked Example: A Simulated Nonlinear-Model Residual Pair

Suppose a hypothetical simulation uses a nonlinear concentration model, but for one individual its predictions are locally approximated by a linearization. The fitted conditional residuals are \(r=(0.42,-0.80)^\mathsf{T}\) concentration units, and the local conditional residual covariance is \(S=\operatorname{diag}(0.49,0.64)\) concentration-squared units. These quantities are simulated, not observations from a real study.

Because \(S\) is diagonal, its Cholesky factor is \(L=\operatorname{diag}(0.70,0.80)\) concentration units. Applying \(CWRES=L^{-1}r\) gives

CWRES= \begin{pmatrix}0.42/0.70\\-0.80/0.80\end{pmatrix} = \begin{pmatrix}0.600\\-1.000\end{pmatrix}.

Each residual is divided by its own conditional standard deviation, so the concentration units cancel. The resulting values are dimensionless and individually compatible with ordinary variation under a standard-normal reference. This pair alone cannot show whether the nonlinear model’s linearization is adequate: that judgment also depends on the curvature of the prediction function over plausible individual effects and on how well those effects are informed by the observations.

When the Approximation Weakens

CWRES are most interpretable when the model’s local approximation and variance structure represent the data reasonably well. Several conditions deserve particular attention:

Limited nonlinear curvature. If the prediction changes substantially nonlinearly over plausible values of \(\eta_i\), a single local sensitivity matrix \(G\) may not capture how individual effects alter predictions. The linear-Gaussian derivation then describes the approximation, not the full nonlinear model.
Informative observations for individual effects. With sparse sampling, individual effects can be weakly identified by that individual’s data. Conditional estimates and the local covariance approximation may then be sensitive to assumptions or numerical details. Sparsity alone does not invalidate the exact linear Gaussian result, but it can make the nonlinear approximation and estimated conditional quantities less dependable.
Appropriate residual distribution and variance. The derivation assumes the residual covariance \(R\) is correctly specified. If the error is non-normal, heteroscedastic in an unmodeled way, or evaluated on an inappropriate scale, normality of the standardized residuals does not follow.
Reliable estimation and computation. The reference uses known covariance parameters and a well-defined conditional estimate. In practice these are estimated, and optimization, boundary estimates, or an unstable covariance estimate can affect the resulting values.

A local linearization can be inspected by comparing the nonlinear prediction with its linear approximation over a plausible range of individual effects. If the difference is large relative to the residual standard deviation, the approximation may materially affect the residual standardization. Sparse records, poor individual-effect precision, or convergence warnings also merit caution; none alone proves that a model is wrong, but each weakens an unqualified normal interpretation.

What CWRES Add—and What They Do Not

Residual approach	What it accounts for	Important caution
Residuals standardized using \(R\) at individual predictions	Residual-error scale at the conditional prediction	Does not, by itself, provide the conditional covariance correction derived above.
Population-level weighted residuals	Deviation from population predictions with a population-level covariance reference	Answers a different question from residuals formed after conditioning on an individual effect.
CWRES	Conditional residuals standardized using a covariance that accounts for the linearized individual-effect estimation	The standard-normal interpretation remains approximate for nonlinear models and estimated parameters.

These approaches are not interchangeable merely because each produces a weighted residual. CWRES are useful because their conditional covariance correction supports a common reference scale while respecting the model’s individual-effect structure. They do not prove that the structural model, covariates, sampling design, or error model is correct. Nor does a value close to zero certify that a particular observation is well described in every respect.

Common Mistake

Common mistake: “A CWRES is defined to be standard normal, so every value can be interpreted exactly as a draw from \(N(0,1)\).”

Correction: Standard normality is exact in the stated linear Gaussian reference model when its covariance parameters are known and the residual vector is whitened by its true covariance. In population PK applications, the model is generally nonlinear, individual effects and parameters are estimated, and the covariance calculation relies on approximations. Treat the normal distribution as a justified reference only to the extent those assumptions are credible.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

In the linear Gaussian derivation, why does the residual covariance equal \(RV^{-1}R\), rather than simply \(R\)?
A hypothetical scalar case has residual \(r=0.30\) and conditional residual variance \(S=0.09\). Calculate the CWRES and state its units.
Does having only one observation per individual automatically invalidate the standard-normal result? Explain the distinction between the exact reference model and a nonlinear application.
Name two reasons a CWRES value may not follow the standard-normal reference closely, even if it was calculated correctly.

Solutions. (1) The fitted residual is \(r=RV^{-1}e\), while \(\operatorname{Var}(e)=V\). Therefore \(\operatorname{Var}(r)=RV^{-1}VV^{-1}R=RV^{-1}R\); conditioning and fitting the individual effect change the residual covariance. (2) \(CWRES=r/\sqrt{S}=0.30/\sqrt{0.09}=0.30/0.30=1.00\). The value is dimensionless because the residual and its standard deviation have the same units. (3) No. With a correctly specified linear Gaussian model, the derivation can hold even with one observation. In a nonlinear application, sparse data may weakly identify the individual effect and make the local linearization less dependable. (4) Examples include important nonlinear curvature, a misspecified residual variance or distribution, estimated parameters, or unstable numerical estimation.