Why Residual Scale Matters
A PD residual records the discrepancy between an observed response and a model prediction. Its numerical size can mean different things depending on the scale used. A deviation of 2 response units may be minor when observation variability is large and notable when variability is small. It may also be unusually large for an observation whose value strongly influenced the fitted model. Comparing residual scales helps separate these situations.
This tutorial considers continuous PD responses and a model that predicts the response at each observation. To make the comparisons precise, assume initially that observations are conditionally independent given the model and have a Gaussian observation distribution. The mean and variance may depend on time, concentration, or other predictors. Residual plots can still be informative when normality is imperfect, but formal reference interpretations require more caution.
The word “standardized” is not used identically in every software package or modeling context. Here it refers specifically to a Pearson residual with a further adjustment for leverage. In a simple regression with constant variance, the adjustment uses the estimated residual standard deviation and the fitted-value leverage. In nonlinear or mixed-effects PD models, leverage may be difficult to define exactly; a reported standardized residual then depends on the model and software’s stated convention. Always check that convention before comparing values across tools.
Three Scales, Three Questions
The raw residual retains the response’s original units. Its sign is direct: \(e_i>0\) means the observed response exceeds the prediction, and \(e_i<0\) means it falls below. Raw residuals are useful for assessing the direction and size of discrepancies on the clinical or assay scale. But raw magnitudes alone do not account for different expected observation variances.
The Pearson residual divides by the model’s predicted observation standard deviation. Under a correctly specified Gaussian observation model with known parameters, \(e_i/\sigma_i\) has a standard normal distribution before accounting for fitted parameters. In practice, parameters are estimated, and observations may be correlated or affected by random effects, so a Pearson residual is not automatically an independent standard normal value. It is nevertheless useful for comparing discrepancies relative to the model’s expected variability.
A leverage-adjusted standardized residual also accounts, approximately, for the fact that an observation can help determine its own fitted value. In ordinary least squares, a high-leverage observation tends to have a smaller residual variance because the fitted line or surface is pulled toward it. The adjustment can therefore make an influential observation’s discrepancy more visible. It does not remove influence or prove that an observation is an outlier.
| Residual | Scaling | What it is useful for | Key caution |
|---|---|---|---|
| Raw, \(e_i\) | No scaling; response units | Direction and absolute discrepancy on the PD scale | Not comparable across observations with different expected variances |
| Pearson, \(r_i\) | Divide by predicted observation SD | Discrepancy relative to modeled variability | Depends on the variance model and ignores leverage adjustment |
| Standardized, \(r_i^*\) | Pearson residual divided by \(\sqrt{1-h_{ii}}\) | Approximate comparison that accounts for fitted-value leverage | Leverage and residual variance may be only approximate in complex models |
Why the Leverage Adjustment Appears
The adjustment has an exact simple-regression justification. Suppose \(y=X\beta+\varepsilon\), where the errors are independent with mean zero and common variance \(\sigma^2\), and \(X\) is the design matrix. The fitted values are \(Hy\), where \(H=X(X^\mathsf{T}X)^{-1}X^\mathsf{T}\) is the projection matrix. The vector of residuals is therefore \(e=(I-H)y=(I-H)\varepsilon\), since \((I-H)X\beta=0\).
Because \(I-H\) is symmetric and idempotent, the residual covariance matrix is \(\sigma^2(I-H)\). Its \(i\)th diagonal element gives the variance of residual \(i\):
Thus dividing by \(\sigma\sqrt{1-h_{ii}}\) gives a residual with unit variance under this model. In practice, \(\sigma\) is estimated, and the corresponding Pearson-like residual is divided by \(\sqrt{1-h_{ii}}\). This result is exact for the stated linear model with common variance; extensions to heteroscedastic or nonlinear PD models require an appropriate variance and leverage approximation. It does not imply that residuals from the same fitted model are independent.
Worked Example: Same Raw Error, Different Observation Variance
Suppose two hypothetical PD measurements each exceed their predicted mean by 2.0 response units. The model predicts observation standard deviations of 1.0 unit for the first and 4.0 units for the second. For this comparison, assume the fitted mean and variance are treated as fixed and set aside leverage.
For both observations, the raw residual is \(e_i=y_i-\hat{\mu}_i=2.0\) units. The Pearson residuals are:
The raw scale says that both responses are 2.0 units above prediction. The variance-scaled comparison says the first discrepancy is two predicted standard deviations, whereas the second is half of one. The first is more surprising relative to the specified observation variability. This conclusion depends on the variance model being reasonable; an incorrectly large predicted standard deviation can make residuals appear deceptively small.
Worked Example: Adjusting for Leverage
Consider a hypothetical continuous PD regression with estimated residual standard deviation \(s=2.0\) response units. One observation has raw residual \(e_i=2.0\) units and leverage \(h_{ii}=0.75\). Assume constant variance, so its Pearson residual is \(e_i/s\).
The Pearson residual is \(2.0/2.0=1.00\). Its standardized value is:
The observation has a Pearson residual of 1.00 but a leverage-adjusted standardized residual of 2.00. The increase follows from \(\sqrt{1-h_{ii}}=0.500\): a high-leverage point has a reduced residual variance in the fitted linear model. This is a prompt to inspect the observation and the model fit, not proof of an error or a reason to delete the point. The adjustment also depends on a suitable leverage calculation and a credible variance estimate.
Using the Scales to Read a PD Diagnostic
Suppose a continuous response is modeled with a standard deviation that increases with its predicted mean. Raw residuals may grow in absolute size at higher predictions even when the model’s relative error remains similar. Plotting only raw residuals can then make a variance pattern look like a change in model bias. Pearson residuals help assess whether discrepancies are large relative to the variance specified by the model. Conversely, a pattern in Pearson residuals may signal that the variance model itself is inadequate.
Worked Example: Checking for a Remaining Variance Pattern
A hypothetical PD model produces the following residuals at four observation times. The fitted model’s predicted observation standard deviations are shown in the table. For simplicity, assume all four leverage values are \(h_{ii}=0.10\); the variance estimates and fitted predictions are treated as fixed for this calculation.
| Observation | Raw residual \(e_i\) (response units) | Predicted SD \(\hat{\sigma}_i\) (response units) |
|---|---|---|
| 1 | 1.0 | 1.0 |
| 2 | 2.0 | 2.0 |
| 3 | -2.0 | 1.0 |
| 4 | -4.0 | 4.0 |
For observation 1, \(r_1=1.0/1.0=1.00\). For observation 2, \(r_2=2.0/2.0=1.00\). For observation 3, \(r_3=-2.0/1.0=-2.00\). For observation 4, \(r_4=-4.0/4.0=-1.00\). With \(h_{ii}=0.10\), the leverage adjustment divides each Pearson residual by \(\sqrt{0.90}=0.9487\). The standardized residuals are therefore approximately 1.05, 1.05, -2.11, and -1.05.
The raw residual magnitudes range from 1.0 to 4.0 units, but much of that range corresponds to different predicted standard deviations. The scaled values flag observation 3 as the largest negative discrepancy relative to its modeled variability. With only four observations, this is a diagnostic clue, not evidence of a systematic problem. A larger data set and plots against time, predicted response, or relevant covariates would be needed to assess whether negative residuals cluster or whether the variance model has a broader failure.
Assumptions, Diagnostics, and Limitations
Residuals are diagnostic summaries, not independent measurements of model error. A residual’s value depends on the prediction, variance, and possibly individual-level estimates used to construct it. In a population PD analysis, population predictions and individual conditional predictions answer different questions. The residual scale does not make those prediction targets interchangeable; state which prediction produced the residual before interpreting a plot.
Several checks help identify when a residual comparison may be misleading:
- Check units and signs. Raw residuals must be in response units, and positive values must consistently mean observed response above prediction.
- Check the variance scale. Pearson residuals are only as credible as the predicted standard deviations. Very small or very large values of \(\hat{\sigma}_i\) can dominate their interpretation.
- Check leverage and fitting method. A leverage adjustment from ordinary regression may not be appropriate for a nonlinear mixed-effects fit without a suitable definition or approximation.
- Look for structure rather than a single threshold. Trends, clusters, or changing spread against time or prediction can be more informative than counting residuals outside an arbitrary cutoff.
- Consider dependence and distributional assumptions. Serial dependence, non-Gaussian responses, and parameter estimation affect reference interpretations. A residual near 2 is not automatically a formal significance test.
Raw residuals remain valuable when the question concerns the size of error in clinically interpretable units. Pearson residuals are often preferable for comparing observations with different modeled variances. Leverage-adjusted residuals can help identify discrepancies partially hidden by a flexible fit. None is universally best. For a nonlinear or mixed-effects model, residual plots, prediction checks, and simulation-based diagnostics may be more appropriate than relying on a simple normal-reference rule. Those approaches complement rather than replace understanding what a residual was scaled by.
Common Mistake
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- An observation is 3.0 response units above its prediction, and the predicted observation SD is 1.5 units. What are its raw and Pearson residuals?
- Two residuals are both 2.0 units. Their predicted SDs are 1.0 and 5.0 units. Which has the larger Pearson residual, and what are the values?
- In the stated linear-model setting, what happens to the leverage-adjusted standardized residual when \(h_{ii}\) increases while \(e_i\) and \(\sigma\) remain fixed?
- Why can a large raw residual at a high predicted response fail to indicate a larger-than-expected discrepancy?
Solutions. 1. The raw residual is \(+3.0\) units. The Pearson residual is \(3.0/1.5=2.00\), dimensionless. 2. The observation with SD 1.0 has Pearson residual \(2.0/1.0=2.00\); the one with SD 5.0 has \(2.0/5.0=0.400\). The first is larger relative to its modeled variability. 3. Since \(r_i^*=e_i/[\sigma\sqrt{1-h_{ii}}]\), increasing \(h_{ii}\) decreases the denominator and increases the absolute standardized residual, if the other quantities remain fixed. 4. The model may predict larger observation variability at high responses. A large absolute error can still be small relative to the predicted SD, so the variance-scaled residual is needed to assess that comparison.