Deriving Variance-Standardized Residuals for Continuous PD

Why Residuals Need a Variance Scale

A raw PD residual is the difference between an observed response and its model prediction. Its size is expressed in the response’s units, but that size is hard to interpret without knowing how much observation error the model expects. A residual of 1 response unit may be unusual when the expected error is 0.2 units and unremarkable when it is 3 units. Variance-standardizing a residual expresses its size relative to the model’s expected observation-error standard deviation.

This distinction matters when the response is measured across a range of concentrations. If the model allows residual variability to increase with concentration, dividing every residual by one common standard deviation will not put observations on comparable scales. The appropriate scale comes from the variance function evaluated for the observation’s model conditions.

Definition: For observation $i$, let $y_i$ be the observed continuous PD response, $\mu_i$ its model-predicted mean, and $V_i$ the model-based residual variance for that observation. The raw residual is $e_i=y_i-\mu_i$. The variance-standardized residual is $$ r_i^{(V)}=\frac{e_i}{\sqrt{V_i}}. $$ It is dimensionless. In this tutorial, the term means scaling by the model-based observation-error standard deviation; it does not by itself correct for the effects of estimating model parameters.

Derivation From the Error Model

Write a continuous-response observation model as $Y_i=\mu_i+\varepsilon_i$, where $\mu_i$ is the model mean and the error has conditional mean zero and model variance $V_i$:

E(\varepsilon_i)=0,\qquad \operatorname{Var}(\varepsilon_i)=V_i.

For the derivation, treat the mean and variance as specified rather than estimated. Since $Y_i-\mu_i=\varepsilon_i$, scaling by the positive square root of the variance gives

E\left(\frac{Y_i-\mu_i}{\sqrt{V_i}}\right) =\frac{E(\varepsilon_i)}{\sqrt{V_i}}=0, \qquad \operatorname{Var}\left(\frac{Y_i-\mu_i}{\sqrt{V_i}}\right) =\frac{\operatorname{Var}(\varepsilon_i)}{V_i} =\frac{V_i}{V_i}=1.

Thus, when the model’s mean and variance are correct, variance-standardization puts residuals on a common variance scale. This result does not require normally distributed errors: it establishes mean zero and variance one, not a particular distribution. If, in addition, $\varepsilon_i\sim N(0,V_i)$, then the variance-standardized error has a standard normal distribution when the model quantities are known.

In practice, $\mu_i$ and $V_i$ are usually evaluated using fitted parameters. The calculated residual is then $r_i^{(V)}=(y_i-\hat{\mu}_i)/\sqrt{\hat{V}_i}$. Because the same observations may have contributed to fitting the model, these residuals need not be independent or have exactly unit variance. The simple derivation describes the scale implied by the error model; it does not establish an exact reference distribution for residuals from a fitted model.

Specify the observation mean.
For each observation, calculate the model prediction $\mu_i$ using the PD model and the observation’s conditions.

Evaluate the variance function.
Calculate $V_i$ at the same conditions, using the fitted or specified variance parameters.

Form and scale the residual.
Subtract prediction from observation, then divide by $\sqrt{V_i}$, not by $V_i$.

Constant Residual Variance

The simplest continuous-response error model assumes a common residual variance across observations. Under a normal error assumption, it can be written $Y_i=\mu_i+\varepsilon_i$, with $\varepsilon_i\sim N(0,\sigma^2)$. The variance is $V_i=\sigma^2$, so the model-based residual standard deviation is $\sigma$, and $r_i^{(V)}=(y_i-\mu_i)/\sigma$. A common standard deviation is appropriate only if the model’s residual variability is reasonably constant over the conditions being analyzed.

Worked Example: Scaling a Residual Under Constant Variance

Suppose a hypothetical continuous PD model predicts a response of 6.9 response units at one observation. The observed response is 7.8 units, and the model assumes a constant residual standard deviation of 1.5 response units. Calculate and interpret the variance-standardized residual.

The raw residual is $$ e=7.8-6.9=0.9\ \text{response units}. $$ The model variance is $V=1.5^2=2.25\ (\text{response units})^2$, so its square root is 1.5 response units. Therefore, $$ r^{(V)}=\frac{7.8-6.9}{\sqrt{2.25}} =\frac{0.9}{1.5}=0.600. $$

The observed response is 0.600 modeled residual standard deviations above its prediction. The residual remains a model diagnostic, not a formal probability or a standalone test. Its interpretation depends on whether the assumed constant variance is adequate and on how the model was fitted.

Concentration-Dependent Residual Variance

A variance function allows the expected residual spread to change with an observation’s conditions. For example, let $C_i$ be concentration and suppose the residual standard deviation is

\sigma(C_i)=\sqrt{\sigma_0^2+(\kappa C_i)^2}, \qquad V_i=\sigma_0^2+(\kappa C_i)^2.

Here $\sigma_0$ is an additive-error scale in response units, and $\kappa$ has units of response per concentration unit. Thus $\kappa C_i$ is in response units, like $\sigma_0$, and the terms being added in the variance function have squared-response units. The two contributions combine as variances, so the standard deviation is the square root of their sum. This particular function is an illustrative model choice, not a requirement for all PD data.

Worked Example: Standardizing at a Given Concentration

In hypothetical data, a model predicts 5.0 response units at concentration $C=3$ concentration units, and the observation is 6.2 response units. Suppose $\sigma_0=0.4$ response units and $\kappa=0.10$ response units per concentration unit. Use the variance function above.

The raw residual is $e=6.2-5.0=1.2$ response units. At this concentration, the concentration-dependent contribution to the standard deviation is $\kappa C=0.10\times3=0.30$ response units. Therefore, $$ V=0.4^2+0.30^2=0.16+0.09=0.25\ (\text{response units})^2, \qquad \sqrt{V}=0.50\ \text{response units}. $$ The variance-standardized residual is $$ r^{(V)}=\frac{1.2}{\sqrt{0.25}} =\frac{1.2}{0.50}=2.40. $$

The response is 2.40 modeled residual standard deviations above its prediction. This is a relatively large scaled residual under the stated variance model, but that description alone does not establish a specific probability or prove that the model is wrong.

Worked Example: Equal Raw Residuals, Different Scales

Consider two hypothetical observations with the same raw residual, $e=0.8$ response units, but concentrations $C=2$ and $C=6$ concentration units. Let $\sigma_0=0.3$ response units and $\kappa=0.20$ response units per concentration unit. Compare their variance-standardized residuals.

At $C=2$, $\kappa C=0.20\times2=0.40$ response units. The variance and standardized residual are $$ V_2=0.3^2+0.4^2=0.09+0.16=0.25, \qquad r_2^{(V)}=\frac{0.8}{\sqrt{0.25}}=\frac{0.8}{0.5}=1.60. $$ At $C=6$, $\kappa C=0.20\times6=1.20$ response units, giving $$ V_6=0.3^2+1.2^2=0.09+1.44=1.53, \qquad r_6^{(V)}=\frac{0.8}{\sqrt{1.53}}\approx\frac{0.8}{1.237}=0.647. $$

Both observations are 0.8 response units above their predictions, but the model expects more residual variability at the higher concentration. Consequently, the same raw difference is smaller relative to the expected error scale there. This is the reason a common divisor can misrepresent residual size when variance changes with concentration.

Interpreting and Checking the Scale

A positive variance-standardized residual means the observation exceeds its prediction; a negative value means it falls below. Its magnitude describes the discrepancy in units of the model-based residual standard deviation. A value near zero indicates a small discrepancy on that scale, while a large absolute value warrants attention. No single magnitude is a universal cutoff: interpretation depends on model fitting, sample size, the assumed error distribution, and the diagnostic purpose.

Variance-standardization is useful only when the variance function is plausible. Inspect whether the residual scale appears systematically to change across concentration or predicted response. A pattern of increasing spread, for example, may indicate that a constant-variance assumption is inadequate. Conversely, a variance function that predicts excessive spread in some range can make residuals there appear artificially small. Such patterns are evidence to investigate the model, not automatic proof of a particular misspecification.

Check the units. The denominator must be a standard deviation in response units, so the ratio is dimensionless. Dividing by variance instead would leave units of inverse response and would not produce unit variance.
Check the variance calculation. Add terms with compatible squared-response units, and take the square root only after evaluating the variance function.
Check the model’s scale assumption. A residual ratio cannot repair an incorrect mean model or an unsuitable variance function.
Remember parameter estimation. Fitted residuals can be constrained by the fitting process and can be related to one another. Do not assume that every calculated residual is exactly an independent standard normal observation.

Raw residuals are easiest to report in response units and can reveal direction and absolute discrepancy, but do not account for changing residual variability. Variance-standardized residuals account for the model’s observation-error scale and are more comparable across conditions if the variance model is adequate. Other adjustments may account for effects of fitting or use a different probability-based construction; those answer related but distinct diagnostic questions. The central choice here is the variance function: standardization cannot make a poorly specified variance model correct.

Key takeaway: Derive the residual scale from the observation error model: subtract the predicted mean from the observed response, then divide by the square root of the variance function evaluated for that observation. The result is dimensionless and has unit variance under the correctly specified, known model; with fitted quantities, that interpretation is generally approximate.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

If $E(\varepsilon_i)=0$ and $\operatorname{Var}(\varepsilon_i)=V_i$, why does dividing $\varepsilon_i$ by $\sqrt{V_i}$ give unit variance?
A hypothetical observation is 4.4 response units, its prediction is 5.0 response units, and its residual standard deviation is 0.5 response units. Calculate and interpret its variance-standardized residual.
For $\sigma(C)=\sqrt{\sigma_0^2+(\kappa C)^2}$, let $\sigma_0=0.6$ response units, $\kappa=0.15$ response units per concentration unit, and $C=4$ concentration units. If the raw residual is $-0.9$ response units, calculate the variance-standardized residual.
Does a variance-standardized residual have to follow a standard normal distribution? State the assumptions needed for that conclusion when model quantities are known.

Solutions. (1) Scaling a random variable by a constant divides its variance by the square of that constant, so $\operatorname{Var}(\varepsilon_i/\sqrt{V_i})=V_i/V_i=1$. (2) $r^{(V)}=(4.4-5.0)/0.5=-0.6/0.5=-1.20$; the observation is 1.20 modeled residual standard deviations below its prediction. (3) $\kappa C=0.15\times4=0.60$ response units, so $V=0.6^2+0.6^2=0.72$ squared response units and $\sqrt{V}=\sqrt{0.72}\approx0.8485$ response units. Thus $r^{(V)}=-0.9/\sqrt{0.72}\approx-1.061$, about 1.061 modeled residual standard deviations below prediction. (4) No. Unit variance follows from the mean-zero and variance assumptions; standard normality additionally requires normally distributed errors with the specified variance. With fitted model quantities, residuals need not have that exact distribution even under a normal observation model.