Why Residual Error Matters Across a Saturating Profile
A saturable elimination model can produce concentrations that span a wide range over time. The structural model describes the expected concentration trajectory; the residual-error model describes how measured concentrations vary around that trajectory. These are different parts of the model. A suitable saturable elimination law cannot compensate reliably for an error model that assigns implausible variability to the observations.
For a typical concentration assay, variability may be roughly constant in absolute concentration units, roughly constant as a percentage, or contain both features. The choice changes how observations at different concentration levels contribute to estimation. It can therefore affect estimates of the structural parameters, including \(V_{\max}\) and \(K_m\), even though it does not change the form of the elimination law.
The usual Gaussian residual models assume that observations are conditionally centered on their predictions and are independent given the model parameters and random effects. In the examples below, the structural predictions are treated as given when discussing residuals. This conditional framing separates residual variability from between-individual variability represented by \(\eta_i\). It does not imply that the marginal distribution of observations, after averaging over random effects, is Gaussian.
Three Common Error Models
In an additive model, the residual has constant standard deviation in concentration units. In a proportional model, its standard deviation is proportional to the prediction, so the coefficient of variation is constant. A combined model includes both sources of variability.
For the combined model, the additive and proportional error components are assumed independent, centered Gaussian variables, with standard deviations \(\sigma_{\mathrm{add}}\) and \(\sigma_{\mathrm{prop}}|f_{ij}|\). The proportional coefficient \(\sigma_{\mathrm{prop}}\) is dimensionless; \(\sigma_{\mathrm{add}}\) has the same units as concentration. The variance formula follows by adding the variances of independent components.
The Gaussian proportional form is a convenient way to express constant relative variability, but it can assign positive probability to negative concentrations. This can be implausible when the proportional error is large or predictions are small. A lognormal observation model is one alternative when strictly positive observations and multiplicative variability are appropriate, but it is a different distributional assumption and requires care about whether the structural prediction represents a mean or a median.
Worked Example: Comparing Error Scales Across Concentrations
Suppose hypothetical predicted concentrations are 100, 10, and 1 mg/L. Compare an additive model with \(\sigma_{\mathrm{add}}=2\) mg/L, a proportional model with \(\sigma_{\mathrm{prop}}=0.20\), and a combined model with \(\sigma_{\mathrm{add}}=1.5\) mg/L and \(\sigma_{\mathrm{prop}}=0.15\).
Under the additive model, the residual standard deviation is 2 mg/L at each prediction. Under the proportional model, it is \(0.20f\): respectively 20, 2, and 0.20 mg/L. Under the combined model, it is \(\sqrt{1.5^2+(0.15f)^2}\).
| Prediction (mg/L) | Additive SD (mg/L) | Proportional SD (mg/L) | Combined SD (mg/L) |
|---|---|---|---|
| 100 | 2.00 | 20.0 | \(\sqrt{2.25+225}=15.1\) |
| 10 | 2.00 | 2.00 | \(\sqrt{2.25+2.25}=2.12\) |
| 1 | 2.00 | 0.200 | \(\sqrt{2.25+0.0225}=1.51\) |
The additive model gives the same absolute spread throughout. The proportional model gives the same relative spread—20%—at every prediction. In the combined model, proportional variability dominates at 100 mg/L, while the additive component is more influential at 1 mg/L. The concentration level alone does not establish which assumption is right; the assay and observed residual behavior must support it.
Likelihood and the Weight Given to Each Observation
For any of these Gaussian models, write \(V_{ij}\) for the conditional residual variance. Given the individual parameters, the contribution of one observation to the log likelihood is
The squared residual is scaled by its variance, but the likelihood also includes the term \(\log(V_{ij})\). That term matters: increasing the assumed variance reduces the squared-residual penalty but also pays a likelihood penalty for spreading probability over a wider range. In an NLME model, the conditional likelihood is combined with the random-effects distribution and integrated over the random effects to form the marginal likelihood. The formula above explains the observation-level contribution conditional on those effects; it is not the whole population likelihood.
This variance scaling explains the practical difference among the models. Additive error uses the same variance weight at all predictions. Proportional error uses larger absolute variance at larger predictions, so a residual of a given size in concentration units is less surprising there. Combined error changes smoothly between an additive-dominated low range and a proportional-dominated high range.
Worked Example: Standardizing Residuals Under Different Models
Consider two hypothetical observations: at a prediction of 4 mg/L, the observed value is 5.2 mg/L; at a prediction of 40 mg/L, it is 44 mg/L. The residuals are \(1.2\) and \(4\) mg/L. Compare an additive model with \(\sigma_{\mathrm{add}}=2\) mg/L, a proportional model with \(\sigma_{\mathrm{prop}}=0.10\), and a combined model with \(\sigma_{\mathrm{add}}=1\) mg/L and \(\sigma_{\mathrm{prop}}=0.10\).
A standardized residual is \(z_{ij}=r_{ij}/\sqrt{V_{ij}}\). Under the additive model, the standard deviation is 2 mg/L at both predictions, giving \(z=1.2/2=0.600\) and \(z=4/2=2.00\).
Under the proportional model, the standard deviations are \(0.10(4)=0.400\) mg/L and \(0.10(40)=4.00\) mg/L. The standardized residuals are \(1.2/0.400=3.00\) and \(4/4.00=1.00\).
Under the combined model, the standard deviations are \(\sqrt{1^2+(0.10\times4)^2}=\sqrt{1.16}=1.077\) mg/L and \(\sqrt{1^2+(0.10\times40)^2}=\sqrt{17}=4.123\) mg/L. The standardized residuals are \(1.2/1.077=1.11\) and \(4/4.123=0.970\). Thus, the same observations can look quite different after scaling, because each model asserts a different amount of expected variability at each concentration.
Worked Example: Why the Likelihood Includes a Variance Penalty
Take one hypothetical observation with \(f=10\) mg/L and \(y=13\) mg/L, so \(r=3\) mg/L. Compare an additive model with \(\sigma_{\mathrm{add}}=2\) mg/L, a proportional model with \(\sigma_{\mathrm{prop}}=0.20\), and a combined model with \(\sigma_{\mathrm{add}}=1\) mg/L and \(\sigma_{\mathrm{prop}}=0.10\).
The additive variance is \(V=2^2=4\;(\mathrm{mg/L})^2\). The proportional variance is \(V=(0.20\times10)^2=4\;(\mathrm{mg/L})^2\). Each gives \[ \ell=-\tfrac12[\log(2\pi\times4)+9/4]=-2.737. \] They assign the same likelihood to this observation because their variances happen to match at \(f=10\) mg/L.
The combined variance is \(V=1^2+(0.10\times10)^2=2\;(\mathrm{mg/L})^2\), giving \[ \ell=-\tfrac12[\log(2\pi\times2)+9/2]=-3.516. \] Here the narrower variance makes this 3 mg/L residual more discordant, despite the smaller variance. These one-observation calculations illustrate the likelihood, not a basis for selecting an error model: model assessment requires the full data and comparable structural assumptions.
Choosing and Checking an Error Model
Residual diagnostics should be interpreted in relation to the variance assumption. If an additive model is reasonable, residual spread in concentration units should be broadly stable across predictions. If proportional error is reasonable, relative spread should be broadly stable, and residuals divided by their predicted standard deviations should not show systematic changes in spread. With combined error, the expected standard deviation is \(\sqrt{\sigma_{\mathrm{add}}^2+\sigma_{\mathrm{prop}}^2f^2}\), so neither constant absolute spread nor constant CV is expected over the entire range.
These are checks of the assumed error scale, not proof that the structural model is correct. A pattern in residuals can arise from a misspecified concentration-time trajectory, an unsuitable residual distribution, assay behavior, or dependence between observations. Inspect residuals against predictions and time, and consider whether one error model leaves systematic concentration-dependent spread. In a mixed-effects analysis, residuals based on estimated individual effects may be affected by shrinkage; apparent agreement in such residuals is not by itself sufficient evidence of a good model.
The three Gaussian choices can be compared by fitting them to the same observations with the same structural model and then examining likelihood-based criteria and diagnostics. Raw residual sum of squares alone is not a fair comparison when models assign different variances. The error model should also be scientifically plausible for the measurement process. For example, constant absolute assay error may support an additive component, while variability that scales with concentration may support a proportional component. The combined model is flexible, but that flexibility can be weakly identified if the data cover only a narrow concentration range.
- Additive error: useful when absolute measurement uncertainty is approximately stable; it can imply very large relative errors near zero and can allow negative Gaussian observations.
- Proportional error: useful when variability is approximately a constant fraction of the prediction; its variance approaches zero as the prediction approaches zero and may be unsuitable for low concentrations.
- Combined error: can represent both low-range absolute variability and high-range relative variability; it estimates more error parameters and may be poorly determined without adequate concentration coverage.
Concentrations that are below the quantification limit require attention to their censoring status rather than automatic treatment as ordinary measured values under one of these Gaussian models. The residual model describes variability for observations modeled as measured; it does not replace an appropriate method for censored data.
Common Pitfall
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- For a proportional model with \(\sigma_{\mathrm{prop}}=0.15\), what is the residual SD at predicted concentrations of 20 and 2 mg/L?
- A combined model has \(\sigma_{\mathrm{add}}=1.2\) mg/L and \(\sigma_{\mathrm{prop}}=0.10\). Find the residual SD at a prediction of 8 mg/L.
- Why can a proportional Gaussian model be questionable when predictions are close to zero?
- Under an additive model with SD 3 mg/L, what standardized residual corresponds to an observed-minus-predicted residual of \(-4.5\) mg/L?
- Why is comparing raw residual sums of squares alone inappropriate for selecting among these error models?
Solutions. (1) The SD is \(0.15(20)=3.00\) mg/L at 20 mg/L and \(0.15(2)=0.300\) mg/L at 2 mg/L. (2) The variance is \(1.2^2+(0.10\times8)^2=1.44+0.64=2.08\;(\mathrm{mg/L})^2\), so the SD is \(\sqrt{2.08}=1.44\) mg/L. (3) Its variance approaches zero with the prediction, while a Gaussian distribution can also assign probability to negative concentrations. (4) The standardized residual is \(-4.5/3=-1.50\). (5) The models use different variances to scale residuals and likelihood contributions; a fair comparison must account for both the scaled residual term and the log-variance term, along with diagnostics and scientific plausibility.