Why Standardize a Residual?
A raw residual is the difference between an observation and a prediction. Its size is difficult to interpret without knowing how much observation-level variation the model expects at that prediction. A difference of 1 mg/L may be large when the expected observation error is 0.2 mg/L, but small when it is 5 mg/L. Standardization expresses the difference in units of its model-based standard deviation.
The essential choice is not just a denominator: it is the prediction target whose error is being assessed. A residual from a population prediction and one from an individual prediction need not have the same variance. The earlier material on raw, weighted, and conditional residuals distinguishes these prediction targets; here the focus is how their model-based variances determine a standardized scale.
The variance \(V_{ij}\) must match the error being standardized. If it is the conditional observation-error variance, the residual is standardized relative to observation noise conditional on the model’s mean, covariates, and any specified individual effects. It is not automatically the variance of a future observation around a population prediction. That broader predictive variance may also include between-individual variation.
Deriving the Standardization
Suppose an observation follows the NLME observation model
where \(f_{ij}(\theta,\eta_i)\) is the model mean for the specified individual and occasion, \(\theta\) denotes population parameters, \(\eta_i\) denotes individual effects, and \(\epsilon_{ij}\) is observation error. Conditional on the prediction target and covariates, assume the observation error has mean zero and variance \(R_{ij}\):
If the prediction is \(f_{ij}(\theta,\eta_i)\), then the prediction error is \(\epsilon_{ij}\). Since \(R_{ij}\) is the conditional variance of that error, dividing by its standard deviation gives a dimensionless quantity. With the model and variance treated as known,
Thus, standardization removes the modeled change in residual scale across observations. The result has conditional mean zero and variance one under these assumptions. This does not by itself imply a normal distribution: that requires an additional distributional assumption about the observation error. In practice, parameters and individual effects are estimated, so the calculated residual is usually a plug-in quantity rather than a value standardized using known parameters.
How Observation-Error Models Set the Denominator
The observation-error model determines how the expected residual variance changes with the prediction. For simplicity, write \(f_{ij}\) for the relevant model prediction. The following formulas concern the conditional observation variance, with the prediction and any specified individual effects held fixed.
| Observation-error model | Conditional error variance | Standardizing denominator |
|---|---|---|
| Additive: \(Y=f+\epsilon_a\) | \(\sigma_a^2\) | \(\sigma_a\) |
| Proportional: \(Y=f(1+\epsilon_p)\) | \(f^2\sigma_p^2\) | \(|f|\sigma_p\) |
| Combined independent additive and proportional errors | \(\sigma_a^2+f^2\sigma_p^2\) | \(\sqrt{\sigma_a^2+f^2\sigma_p^2}\) |
Here \(\epsilon_a\) has mean zero and variance \(\sigma_a^2\), while \(\epsilon_p\) has mean zero and variance \(\sigma_p^2\). In the combined model they are assumed independent, so the variance of their sum is the sum of their variances. The additive scale \(\sigma_a\) has the units of \(Y\). The proportional scale \(\sigma_p\) is dimensionless, and its contribution to the observation-error standard deviation grows with the magnitude of \(f\).
Worked Example: Standardizing Under Additive Error
Suppose a hypothetical concentration observation is \(8.4\) mg/L and the matching individual prediction is \(7.6\) mg/L. The fitted additive observation-error model has standard deviation \(\sigma_a=0.8\) mg/L. Assume that, conditional on this prediction target, the error has mean zero and constant variance \(\sigma_a^2\).
The raw prediction error is $$ e=8.4-7.6=0.8\text{ mg/L}. $$ The model-based prediction-error variance is \((0.8\text{ mg/L})^2=0.64\text{ (mg/L)}^2\), so its standard deviation is \(0.8\) mg/L. Therefore, $$ r=\frac{8.4-7.6}{0.8}=\frac{0.8\text{ mg/L}}{0.8\text{ mg/L}}=1.00. $$ The observation is one modeled observation-error standard deviation above the individual prediction. The result is dimensionless; it does not establish that the observation is unusual without a reference distribution and the relevant assumptions.
Worked Example: Proportional Error Makes the Scale Prediction-Dependent
For a hypothetical assay result, suppose the prediction is \(20\) mg/L, the observed value is \(23\) mg/L, and the proportional error has standard deviation \(\sigma_p=0.10\). Assume \(Y=f(1+\epsilon_p)\), with mean-zero proportional error of constant variance and no additive component.
The prediction error is \(23-20=3\) mg/L. Its model-based standard deviation is not \(0.10\) mg/L: the proportional scale must be multiplied by the prediction: $$ \sqrt{V}=|f|\sigma_p=(20\text{ mg/L})(0.10)=2.0\text{ mg/L}. $$ Thus, $$ r=\frac{23-20}{2.0}=\frac{3\text{ mg/L}}{2.0\text{ mg/L}}=1.50. $$ The observation is 1.50 modeled observation-error standard deviations above the prediction. Under this model, a larger prediction would imply a larger absolute residual standard deviation, even if the proportional error parameter stayed at 0.10.
Worked Example: Combined Error and the Choice of Prediction Target
Consider a hypothetical measurement with prediction \(f=20\) mg/L and observation \(Y=23\) mg/L. Suppose the model has independent additive error with \(\sigma_a=1.0\) mg/L and proportional error with \(\sigma_p=0.10\). First standardize the residual conditional on the specified individual prediction. Then compare it with a population-level predictive scale in a simple random-intercept model.
For the conditional observation error, the variance is the sum of the additive and proportional variances: $$ V_{\text{obs}}=(1.0\text{ mg/L})^2+(20\text{ mg/L}\times0.10)^2 =1+4=5\text{ (mg/L)}^2. $$ The standard deviation is \(\sqrt{5}=2.236\) mg/L. The error is \(23-20=3\) mg/L, so the conditionally standardized residual is $$ r_{\text{cond}}=\frac{3}{\sqrt{5}}=1.342. $$ Now suppose the population prediction is \(f_0=20\) mg/L, a random individual effect adds \(\eta_i\sim N(0,(3\text{ mg/L})^2)\), and additive observation error has standard deviation \(4\) mg/L. In this separate simple model, \(Y=f_0+\eta_i+\epsilon_i\), with independent \(\eta_i\) and \(\epsilon_i\). The population-level predictive variance is \(3^2+4^2=25\text{ (mg/L)}^2\), giving a predictive standard deviation of \(5\) mg/L. The same observed difference of \(3\) mg/L from the population prediction is \(3/5=0.600\) predictive standard deviations.
These values answer different questions. The first scales the error around an individual prediction by observation noise. The second scales an observation around the population prediction while including between-individual variability. A denominator should not be switched between these targets without changing the interpretation.
Population Prediction Variance Is Not Just Observation Variance
The distinction in the last example follows from the law of total variance. Let \(M_i=f_{ij}(\theta,\eta_i)\) be the model mean when individual effects vary across the population, and let the conditional observation variance be \(R_{ij}(\eta_i)\). Then
The first term is variation in model means across individuals; the second is average observation-error variance. This expression gives variance around the marginal mean \(E_{\eta_i}(M_i)\). If a residual is instead formed around the typical individual prediction \(f_{ij}(\theta,0)\), its mean need not be zero in a nonlinear model. Its mean squared prediction error is
The final term is squared bias relative to the typical prediction. It can be nonzero because, for a nonlinear function, the mean of \(f(\theta,\eta_i)\) need not equal \(f(\theta,0)\). Consequently, a population prediction-error variance cannot always be obtained by adding random-effect variance and observation variance alone; the center of the error must also be checked.
Assumptions, Diagnostics, and Common Pitfalls
Before interpreting a standardized residual, verify that its numerator and denominator refer to the same model target. The prediction should be computed under the stated conditioning, and the variance formula should come from the fitted observation-error model at that prediction. For the additive-proportional formula, the two error components must be independent; if they are correlated, a covariance term is required. If the fitted error variance changes with time, covariates, or prediction, use its value for that observation rather than a single global scale.
- Check scale and units. The prediction-error variance has squared observation units, and its square root must have the same units as the raw residual. The standardized residual should be dimensionless.
- Check the error model at low and high predictions. Under proportional error, the modeled standard deviation approaches zero as the prediction approaches zero. If the assay has non-negligible absolute noise near zero, a purely proportional model may understate the variance there.
- Check whether the prediction is conditional or population-level. Scaling a population residual only by observation-error SD ignores individual variability. Conversely, adding random-effect variability to an error around a conditional individual prediction changes the target.
- Remember that fitted quantities are uncertain. Replacing unknown parameters and individual effects with estimates gives a plug-in standardization. The resulting residuals need not have exactly mean zero or variance one in a finite sample.
- Do not infer a distribution from the scale alone. A zero-mean, unit-variance standardized error is not necessarily normally distributed, independent across observations, or free of systematic structure.
A frequent mistake is to divide every residual by one reported residual standard deviation and call the result standardized. That is justified for a constant-variance additive model, but not generally for a proportional or combined model. For example, using a constant \(2\) mg/L denominator for observations predicted at \(5\) and \(20\) mg/L ignores the changing scale under proportional error. The correction is to calculate the variance from the fitted error model separately at each relevant prediction, then take its square root.
Standardization is therefore a scale adjustment, not a repair for a wrong mean model or wrong error model. A systematic mean mismatch remains in the numerator, and an incorrectly specified variance produces residuals whose apparent scale is misleading. Standardized values can make observations on different modeled scales more comparable, but they do not by themselves validate the model or replace diagnostics of residual patterns.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A hypothetical observation is \(12.5\) mg/L, its matching prediction is \(11.0\) mg/L, and additive observation-error SD is \(0.75\) mg/L. What is the standardized residual?
- Under a proportional error model, \(f=8\) mg/L and \(\sigma_p=0.15\). What is the observation-error SD?
- Why does combining independent additive and proportional errors require adding their variances rather than their standard deviations?
- In one sentence, distinguish conditional observation-error variance from population-level predictive variance.
Solutions. (1) \(r=(12.5-11.0)/0.75=2.00\), so the observation is 2.00 additive error SDs above the prediction. (2) \(|f|\sigma_p=(8)(0.15)=1.20\) mg/L. (3) Independent variances add, so \(V=\sigma_a^2+f^2\sigma_p^2\), and the combined SD is the square root of that sum. (4) Conditional observation-error variance describes noise around a specified individual prediction, whereas population predictive variance also accounts for variation in model means across individuals.