Common Errors in Interpreting SDE Residuals

Why Residuals Need Careful Interpretation

A residual is not automatically a direct measure of model error. In a pharmacokinetic stochastic differential equation (SDE), the state is latent, observations may be noisy, predictions have uncertainty, observation times may be uneven, and errors may be dependent. A diagnostic that ignores any of these features can label ordinary stochastic variation as model failure—or make a real problem look harmless.

The earlier material on one-step predictive distributions defines the relevant starting point: predict the next observation using information available before that observation, then compare the observation with that predictive distribution. This tutorial focuses on common interpretation errors. It does not treat a fitted latent state as if it were observed, nor does it regard several correlated residuals as several independent pieces of evidence.

Definition: Let $Y_i$ be the observation at time $t_i$, and let $\mathcal{F}_{i-1}$ denote the information available immediately before observing $Y_i$, including earlier observations, dosing history, and known covariates. The one-step predictive mean and variance are $m_i=E[Y_i\mid\mathcal{F}_{i-1}]$ and $S_i=\operatorname{Var}(Y_i\mid\mathcal{F}_{i-1})$. The one-step innovation is $r_i=Y_i-m_i$; when $0<S_i<\infty$, its standardized form is $z_i=r_i/\sqrt{S_i}$.

The predictive quantities must match the target being diagnosed. A conditional prediction for one subject may condition on that subject’s estimated random effects; a population prediction may integrate over their distribution. Likewise, a plug-in prediction that treats fitted parameters as fixed does not include parameter-estimation uncertainty. State clearly which prediction is being used before interpreting its residuals.

Latent-State Estimates Are Not Observations

An SDE state $X_{t_i}$, such as a compartment amount or concentration, is generally not observed directly. An observation might instead satisfy $Y_i=h(X_{t_i})+\epsilon_i$, where $h$ is the observation function and $\epsilon_i$ is measurement error. A filtered state estimate uses data through time $t_i$; a predicted state estimate uses only data through $t_{i-1}$. Neither is the observed value itself.

Subtracting a predicted observation from $Y_i$ produces an observation innovation. Subtracting a state estimate from a state or observation is a different quantity, with a different uncertainty and often a different diagnostic interpretation. In particular, a smoothed state estimate uses future observations as well as past ones. Its difference from a nearby observation is therefore not a one-step predictive residual.

Worked Example: A Noisy Observation of an Uncertain State

Before time $t_i$, suppose the predicted latent concentration is $X_{t_i}\mid\mathcal{F}_{i-1}\sim N(10,4)$, and the observation is $Y_i=X_{t_i}+\epsilon_i$. Given the state and past data, let $\epsilon_i$ have mean zero and variance $1$. An observed concentration is $y_i=13$.

The predicted observation mean is $E[Y_i\mid\mathcal{F}_{i-1}]=10$. The state uncertainty contributes variance $4$, and measurement error contributes variance $1$, so the predictive variance is $S_i=5$. Thus $r_i=13-10=3$, and $z_i=3/\sqrt{5}\approx1.342$.

If the predicted state mean $10$ is incorrectly treated as an exact observation of the state, one might divide the same difference $3$ by the measurement-error standard deviation $1$, obtaining $3$. That calculation discards the state uncertainty and does not describe the one-step predictive discrepancy. The correct standardized innovation is about $1.34$, under the stated model.

Prediction Uncertainty Includes State and Observation Uncertainty

The following variance decomposition makes explicit what is lost when only measurement error is used to standardize a residual. Assume that, conditional on $X_{t_i}$ and $\mathcal{F}_{i-1}$, the observation error has mean zero and variance $R_i(X_{t_i})$, where $R_i$ is finite. The error variance is allowed to depend on the latent state. Assume also that $h(X_{t_i})$ has finite second moment.

Result: Under these assumptions, for $Y_i=h(X_{t_i})+\epsilon_i$, $$ E[Y_i\mid\mathcal{F}_{i-1}]=E[h(X_{t_i})\mid\mathcal{F}_{i-1}], \qquad \operatorname{Var}(Y_i\mid\mathcal{F}_{i-1}) =\operatorname{Var}(h(X_{t_i})\mid\mathcal{F}_{i-1}) +E[R_i(X_{t_i})\mid\mathcal{F}_{i-1}]. $$

To verify the result, condition first on the state and past. The zero conditional mean gives $E[\epsilon_i\mid X_{t_i},\mathcal{F}_{i-1}]=0$. Taking conditional expectations again yields $E[\epsilon_i\mid\mathcal{F}_{i-1}]=0$, and hence the stated predictive mean. For the variance, apply the law of total variance:

\operatorname{Var}(Y_i\mid\mathcal{F}_{i-1}) =E[\operatorname{Var}(Y_i\mid X_{t_i},\mathcal{F}_{i-1})\mid\mathcal{F}_{i-1}] +\operatorname{Var}(E[Y_i\mid X_{t_i},\mathcal{F}_{i-1}]\mid\mathcal{F}_{i-1}).

The first term is $E[R_i(X_{t_i})\mid\mathcal{F}_{i-1}]$, and the second is $\operatorname{Var}(h(X_{t_i})\mid\mathcal{F}_{i-1})$, proving the formula. Thus state-dependent observation-error variance does not invalidate the decomposition; it is averaged over the predictive distribution of the state. The zero conditional mean is important. If instead $E[\epsilon_i\mid X_{t_i},\mathcal{F}_{i-1}]=a_i(X_{t_i})$, and the conditional error variance is $V_i(X_{t_i})$, then the predictive mean is $E[h(X_{t_i})+a_i(X_{t_i})\mid\mathcal{F}_{i-1}]$, and the predictive variance is

E[V_i(X_{t_i})\mid\mathcal{F}_{i-1}] +\operatorname{Var}(h(X_{t_i})+a_i(X_{t_i})\mid\mathcal{F}_{i-1}).

This follows from the same law of total variance, using the actual conditional mean and variance of the error. Merely noting dependence between error variance and state is not grounds for changing the zero-mean formula.

Worked Example: State-Dependent Observation-Error Variance

Suppose $h(X)=X$, $E[X_{t_i}\mid\mathcal{F}_{i-1}]=2$, and $\operatorname{Var}(X_{t_i}\mid\mathcal{F}_{i-1})=4$. Conditional on the state and past, let $\epsilon_i$ have mean zero and variance $R_i(X_{t_i})=0.01X_{t_i}^2$. Since $E[X_{t_i}^2\mid\mathcal{F}_{i-1}]=4+2^2=8$, the expected observation-error variance is $0.01(8)=0.08$. Therefore $E[Y_i\mid\mathcal{F}_{i-1}]=2$ and $\operatorname{Var}(Y_i\mid\mathcal{F}_{i-1})=4+0.08=4.08$. The error variance depends on the state, but the stated zero-mean decomposition still applies.

If the conditional error mean were instead $a_i(X)=0.1X$, then the predictive mean would be $E[X+0.1X\mid\mathcal{F}_{i-1}]=2.2$, not $2$. The variance would include $\operatorname{Var}(1.1X\mid\mathcal{F}_{i-1})=1.21(4)=4.84$, as well as the expected conditional error variance. The zero-mean formula cannot be used unchanged in that case.

Unequal Observation Intervals Change Residual Scale

A diffusion-driven state can accumulate more process variation over a longer interval. Consequently, raw residuals from short and long intervals need not have the same scale, even when the model is correct. For a simple illustration, suppose the predicted state follows $dX_t=\sigma\,dW_t$ over an interval of length $\Delta_i=t_i-t_{i-1}$, with known starting state and noiseless observations. The increment has mean zero and variance $\sigma^2\Delta_i$. With independent measurement error of variance $R$, the observation innovation variance is $\sigma^2\Delta_i+R$.

Worked Example: Comparing Innovations Across Unequal Intervals

Let $\sigma=0.5$ concentration units per square-root time unit, and let the independent measurement-error variance be $R=0.09$. For an interval $\Delta_1=0.25$, the predictive variance is $0.5^2(0.25)+0.09=0.0625+0.09=0.1525$, with standard deviation $\sqrt{0.1525}\approx0.391$. For $\Delta_2=4$, it is $0.5^2(4)+0.09=1+0.09=1.09$, with standard deviation $\sqrt{1.09}\approx1.044$.

Suppose the two observed innovations are $r_1=0.4$ and $r_2=1.0$. Their raw magnitudes are $0.4$ and $1.0$, but the standardized values are $z_1=0.4/0.391\approx1.02$ and $z_2=1.0/1.044\approx0.96$. They are similar relative to their respective predictive uncertainty. Calling the second discrepancy more concerning solely because its raw value is larger would ignore the longer interval and its larger expected variation.

The square-root interval rule is only a simple benchmark for constant diffusion with the stated observation model. In a general PK SDE, drift, state-dependent diffusion, nonlinear observation functions, and uncertain initial states can all affect the predictive variance. Use the model’s one-step predictive variance rather than mechanically dividing by $\sqrt{\Delta_i}$. If observation error dominates, even that simple interval scaling may be a poor approximation.

Correlated Residuals Are Not Independent Evidence

A sequence of residuals can be individually plausible while sharing information. Dependence may arise from a modelled correlation structure, an unmodelled process feature, shared measurement effects, or the way estimates were constructed. Serial-correlation checks for innovation sequences address whether such dependence is present; a separate interpretive error is to count correlated observations as independent confirmation.

Result: If $n$ residuals have common variance $\sigma^2$ and every distinct pair has correlation $\rho$, then their average has variance $$ \operatorname{Var}\!\left(\frac{1}{n}\sum_{i=1}^n r_i\right) =\frac{\sigma^2}{n}\{1+(n-1)\rho\}. $$ This formula assumes the stated common variances and pairwise correlations; it does not require normality.

Indeed, the variance of a sum is the sum of the variances plus twice the sum of distinct pairwise covariances. There are $n$ variance terms, each $\sigma^2$, and $n(n-1)/2$ pairs, each with covariance $\rho\sigma^2$. Dividing by $n^2$ gives

\operatorname{Var}\!\left(\frac{1}{n}\sum_{i=1}^n r_i\right) =\frac{n\sigma^2+n(n-1)\rho\sigma^2}{n^2} =\frac{\sigma^2}{n}\{1+(n-1)\rho\}.

For positive $\rho$, this exceeds the independence-based variance $\sigma^2/n$. Treating correlated residuals as independent therefore overstates how precisely their average is known. The formula concerns variance, not a claim that correlated observations are useless; it shows why their joint dependence must be respected.

Worked Example: Three Correlated Residuals

Suppose three residuals each have variance $1$, and every pair has correlation $0.5$. The variance of their sum is $1+1+1+2(0.5+0.5+0.5)=6$. If the observed residuals are $1,1,1$, their sum is $3$. Under the stated covariance structure, the sum divided by its standard deviation is $3/\sqrt{6}\approx1.225$.

An independence calculation would use variance $3$, giving $3/\sqrt{3}\approx1.732$. That larger value is not justified because the assumed residuals are correlated. This calculation illustrates the effect of dependence; it does not itself establish a reference distribution or a formal test.

Practical Checks and Alternatives

Before interpreting a residual plot or summary, verify what each residual compares, which information was available at prediction time, and what uncertainty its scale includes. A large residual may be a genuine surprise, but it can also arise from a wrong residual definition or an incomplete predictive variance. Conversely, a small residual does not prove that a model is adequate.

Identify the target.
Check whether the quantity is an observation, a latent-state estimate, a filtered estimate, or a smoothed estimate. For one-step diagnostics, use a prediction based only on information available before the observation.

Check the predictive distribution.
Confirm that state uncertainty and observation uncertainty are included, and that the observation-error mean and variance assumptions match the model.

Respect interval length and dependence.
Use the predictive scale for each actual interval. Inspect whether residuals share dependence before treating several values as separate evidence.

Report the residual definition.
State whether residuals are raw innovations, standardized innovations, or another quantity, and whether fitted parameters or random effects are treated as fixed or integrated over.

Raw innovations retain the units of the observations and can be useful for recognizing the direction and absolute size of discrepancies, but they are difficult to compare when predictive scales differ. Standardized innovations account for their one-step predictive standard deviations and are more comparable across times under a correctly specified predictive distribution; standardization does not make them independent, nor does it guarantee a standard normal distribution. State-estimation residuals can help describe state reconstruction, but filtered and smoothed versions answer different questions and should not be substituted for prospective observation innovations.

Diagnostics can also go wrong when the prediction conditions on information that would not have been available in the intended use, when the observation model has a nonzero conditional mean that is ignored, or when uncertainty is omitted by treating estimated quantities as known. These limitations do not make residuals uninformative. They mean that residuals are interpretable only relative to their construction and assumptions.

Key takeaway: Interpret a residual against the predictive distribution that generated it. Distinguish latent states from observations, include the relevant uncertainty, respect unequal intervals, and account for dependence before drawing conclusions from a sequence.

Check Your Understanding

Answer each question, then check the concise solutions below.

A predicted observation has mean $8$, and a new observation is $10$. Is the raw innovation $2$ enough to decide whether the observation is surprising? What additional quantity is needed?
Suppose observation error has conditional mean zero given the latent state and past data, but its variance depends on the state. Does that dependence alone invalidate the predictive variance decomposition? State the relevant contribution from observation error.
In the constant-diffusion example with no measurement error, what happens to the standard deviation of an increment when the interval length is multiplied by $9$?
For four residuals with common variance $1$ and pairwise correlation $0.25$, calculate the variance of their average.
Why is a smoothed-state difference generally not a one-step predictive innovation?

Solutions. (1) No. The predictive variance $S_i$, or equivalently the predictive standard deviation, is needed to assess the discrepancy relative to uncertainty. (2) No. The decomposition remains valid under the stated conditional zero-mean and finite-variance assumptions; the observation-error contribution is $E[R_i(X_{t_i})\mid\mathcal{F}_{i-1}]$. (3) The standard deviation is multiplied by $\sqrt{9}=3$, since the increment variance is proportional to interval length. (4) The variance is $\{1+(4-1)(0.25)\}/4=1.75/4=0.4375$. (5) A smoothed estimate uses future observations, while a one-step predictive innovation compares a new observation with a prediction formed before that observation.