Additive Residual Error and Constant Assay Precision

What Additive Residual Error Represents

A concentration prediction and a measured concentration are not identical, even when the pharmacokinetic model is appropriate. Some of the difference may arise from assay measurement, sample handling, or other unexplained observation-level variation. An additive residual model represents this difference as a quantity added to the model prediction. Its defining feature is that the error has a constant variance on the observation scale, not a variance that grows with the predicted concentration.

This assumption is useful when the assay's absolute measurement scatter is approximately similar across the concentration range. “Constant precision” here means constant absolute precision, expressed in the units of the measured response. It does not mean constant relative error, and it does not guarantee that the assay is unbiased.

Definition: Let $y_{ij}$ be the observation for subject $i$ at observation $j$, and let $f_{ij}(\theta,\eta_i)$ be the model prediction for that observation, given population parameters $\theta$ and subject-specific random effects $\eta_i$. The additive residual error model is $$ y_{ij}=f_{ij}(\theta,\eta_i)+\epsilon_{ij},\qquad \epsilon_{ij}\mid \eta_i \sim N(0,\sigma_{\mathrm{add}}^2). $$ The standard additive-error assumptions are that residuals have conditional mean zero, share the same variance $\sigma_{\mathrm{add}}^2$, and are conditionally independent across observations. The residual standard deviation $\sigma_{\mathrm{add}}$ has the same units as $y_{ij}$; its variance has squared response units.

The conditioning matters. The model prediction already depends on the subject’s random effects. Additive residual error describes the remaining observation-level variation around that prediction; it is not another name for between-subject or between-occasion variability. In the basic model, two observations with different predictions have the same residual variance, provided the assumptions hold.

Consequences on the Observation Scale

Under the additive model, the conditional variance and standard deviation of an observation are

\operatorname{Var}(y_{ij}\mid\eta_i)=\sigma_{\mathrm{add}}^2, \qquad \operatorname{SD}(y_{ij}\mid\eta_i)=\sigma_{\mathrm{add}}.

Thus, a residual standard deviation of $0.20$ mg/L means that the model assigns the same absolute scatter, in mg/L, around predictions at $0.5$ mg/L and $5$ mg/L. It does not mean the same percentage scatter. The ratio of residual standard deviation to prediction is $\sigma_{\mathrm{add}}/f_{ij}$, so that ratio increases as the prediction gets smaller.

The normal assumption also gives the conditional distribution $y_{ij}\mid\eta_i\sim N(f_{ij},\sigma_{\mathrm{add}}^2)$. A normal distribution has support below zero. Therefore, if concentrations are nonnegative, the model can assign some probability to negative measured values, especially when a prediction is small relative to $\sigma_{\mathrm{add}}$. This is a mathematical implication of the model, not a claim that a laboratory must report a negative concentration.

Worked Example: Calculating the Additive Residual Variance

Suppose four hypothetical observations have predictions $1.00,\ 2.00,\ 3.00,$ and $4.00$ mg/L, and measured values $1.10,\ 1.90,\ 3.20,$ and $3.80$ mg/L. Treat the predictions as fixed for this calculation and assume independent, zero-mean additive normal errors with a common variance. Calculate the conditional maximum-likelihood estimate of the residual variance.

The residuals, defined as observation minus prediction, are $0.10,\ -0.10,\ 0.20,$ and $-0.20$ mg/L. Their squared sum is

\mathrm{SSE}=(0.10)^2+(-0.10)^2+(0.20)^2+(-0.20)^2 =0.010+0.010+0.040+0.040=0.100\;(\mathrm{mg/L})^2.

For $n=4$ independent normal residuals around fixed predictions, the conditional likelihood estimate is $\widehat{\sigma}_{\mathrm{add}}^2=\mathrm{SSE}/n$. Hence

\widehat{\sigma}_{\mathrm{add}}^2=\frac{0.100}{4} =0.0250\;(\mathrm{mg/L})^2, \qquad \widehat{\sigma}_{\mathrm{add}}=\sqrt{0.0250} =0.158\;\mathrm{mg/L}.

This calculation estimates one common absolute variance; it does not estimate a separate variance for each prediction. In a fitted NLME model, predictions and other parameters are generally estimated together, so the full likelihood fitting procedure—not this fixed-prediction calculation alone—determines the final estimate.

Why Low Concentrations Need Careful Interpretation

With constant absolute error, the same-sized deviation can be minor relative to a large concentration but substantial relative to a small one. This is important near the lower end of a concentration–time profile: a residual of $0.20$ mg/L is 4% of $5.0$ mg/L, but 40% of $0.50$ mg/L. The absolute precision has not changed; the relative size of the error has.

Worked Example: Relative Scatter at Two Predictions

Assume a hypothetical assay is adequately described by additive normal error with $\sigma_{\mathrm{add}}=0.20$ mg/L. Compare the residual standard deviation as a percentage of two predicted concentrations, $0.50$ mg/L and $5.00$ mg/L.

At $0.50$ mg/L, the ratio is $0.20/0.50=0.40$, or 40%. At $5.00$ mg/L, it is $0.20/5.00=0.040$, or 4.0%. Both observations have the same modeled standard deviation, $0.20$ mg/L. The percentages differ because the denominators differ.

The model also implies a nonzero probability of a negative observation at the lower prediction. For a prediction of $0.10$ mg/L,

P(y_{ij}<0\mid\eta_i) =\Phi\left(\frac{0-0.10}{0.20}\right) =\Phi(-0.50) \approx 0.3085.

Here $\Phi$ is the standard normal cumulative distribution function. Under the stated model, the probability is about 30.9%. This calculation flags a potential mismatch if negative values are impossible or if observations near the assay’s quantification limit are recorded as censored rather than as numerical concentrations. In that setting, the data-collection and likelihood treatment must reflect what was actually observed.

Likelihood Justification and Replicate Measurements

The variance estimate used above follows directly from the normal likelihood when predictions are treated as fixed. Write the residual for observation $k$ as $r_k=y_k-f_k$, for $k=1,\ldots,n$. Conditional independence and the common normal variance give the likelihood

L(\sigma_{\mathrm{add}}^2) =(2\pi\sigma_{\mathrm{add}}^2)^{-n/2} \exp\left(-\frac{\sum_{k=1}^{n}r_k^2}{2\sigma_{\mathrm{add}}^2}\right).

Taking logarithms and differentiating with respect to $\sigma_{\mathrm{add}}^2$ gives

\ell=-\frac{n}{2}\log(2\pi)-\frac{n}{2}\log(\sigma_{\mathrm{add}}^2) -\frac{\mathrm{SSE}}{2\sigma_{\mathrm{add}}^2}, \qquad \frac{\partial\ell}{\partial\sigma_{\mathrm{add}}^2} =-\frac{n}{2\sigma_{\mathrm{add}}^2} +\frac{\mathrm{SSE}}{2(\sigma_{\mathrm{add}}^2)^2}.

Setting the derivative to zero yields $n\sigma_{\mathrm{add}}^2=\mathrm{SSE}$, so the likelihood estimate is $\mathrm{SSE}/n$. The second derivative at this point is negative when $\mathrm{SSE}>0$, confirming a maximum. This derivation assumes fixed predictions and independent normal errors. It is not a general degrees-of-freedom correction for every fitted NLME model.

The same assumptions clarify what happens when replicate measurements are averaged. If $m$ independent replicates share a common true model prediction and each has additive residual variance $\sigma_{\mathrm{add}}^2$, their mean has variance $\sigma_{\mathrm{add}}^2/m$. Independence is essential: shared sample-preparation effects or other common errors do not disappear simply by taking more replicates.

Worked Example: Precision of an Average of Replicates

Suppose four hypothetical independent assay replicates are taken from material with a model prediction of $1.00$ mg/L. The results are $1.10,\ 0.90,\ 1.20,$ and $0.80$ mg/L. Assume each replicate has zero-mean additive error with standard deviation $0.20$ mg/L.

The replicate average is $(1.10+0.90+1.20+0.80)/4=1.00$ mg/L. Under independent errors, its variance is

\operatorname{Var}(\bar{y})=\frac{\sigma_{\mathrm{add}}^2}{m} =\frac{(0.20\;\mathrm{mg/L})^2}{4} =0.0100\;(\mathrm{mg/L})^2, \qquad \operatorname{SD}(\bar{y})=0.100\;\mathrm{mg/L}.

The individual replicate standard deviation remains $0.20$ mg/L; the $0.100$ mg/L value describes the standard deviation of averages across repeated sets of four independent replicates. It is not a claim that averaging removes subject-level or other shared variability.

Assumptions, Diagnostics, and Alternatives

Additive residual error is a useful baseline, but its adequacy depends on the observation process and the pattern of deviations after accounting for the structural model and random effects. Evidence against the simple assumptions can include residual spread that systematically changes with predicted concentration, repeated runs of residuals in the same direction, or frequent extreme residuals relative to the fitted standard deviation. Such patterns do not by themselves identify the cause: an inadequate concentration–time model, dependence between observations, or an unsuitable residual distribution may also contribute.

A practical assessment should therefore ask whether the residual scale is roughly stable in response units across the prediction range, whether residuals are centered near zero, and whether independence and approximate normality are plausible for the intended likelihood. In particular, observations near an assay limit deserve attention: censoring, rounding, or a lower bound on reported values can conflict with treating every recorded value as an uncensored normal observation.

The main alternatives differ in what they assume about how residual spread changes. A concentration-dependent structure may be more appropriate when absolute scatter grows with the prediction. A transformed-scale model may be useful when variation is naturally expressed on that scale, but its interpretation is not the same as constant additive error on the observation scale. Correlated residual structures address dependence rather than simply changing the variance level. These are alternatives to assess against the design and assay behavior; they are not interchangeable corrections for every poor fit.

Question	Additive-error implication	Potential warning
How does absolute spread change with prediction?	It remains $\sigma_{\mathrm{add}}$ in response units.	Spread clearly rises or falls across predictions.
How does relative spread change?	It is larger when the prediction is smaller.	A constant relative spread appears more plausible than constant absolute spread.
Can a modeled observation be negative?	Yes, under the normal distribution.	Negative values are impossible, or near-limit data are censored.
Are residuals independent?	Conditional residual covariance between distinct observations is zero.	Residuals show within-subject dependence or shared assay effects.

A common mistake is to say, “The assay has a 20% error because the additive standard deviation is $0.20$ mg/L.” That statement confuses absolute and relative scales. At a prediction of $1.0$ mg/L, the standard deviation is 20% of the prediction; at $5.0$ mg/L, it is 4%. The correction is to report $0.20$ mg/L as the additive residual standard deviation and calculate a relative quantity only for a specified prediction.

Another pitfall is to interpret every residual as assay noise. Residual error is the observation-level component remaining in the specified model; if the structural prediction is systematically wrong, the residuals can reflect that misspecification as well. A fitted additive variance does not prove that the assay itself has constant variance, nor does it establish that the structural model is adequate.

Key takeaway: Additive residual error means constant conditional variance and constant absolute standard deviation on the observation scale. At low predictions, that same absolute scatter can be large relative scatter and can imply appreciable probability below zero under a normal model.

Check Your Understanding

Answer each question, then compare with the solutions below.

If $\sigma_{\mathrm{add}}=0.15$ mg/L, what are the conditional residual variance and standard deviation at predictions of $0.30$ and $3.0$ mg/L?
For $n=3$ residuals $0.10,\ -0.10,\ 0.20$ mg/L around fixed predictions, calculate the conditional likelihood estimate of $\sigma_{\mathrm{add}}^2$ and $\sigma_{\mathrm{add}}$.
Under an additive model with $\sigma_{\mathrm{add}}=0.10$ mg/L, what fraction of the prediction is one residual standard deviation when the prediction is $0.50$ mg/L?
State one reason why the probability of a negative value from an additive normal model may be a concern near an assay limit.

Solutions. (1) At either prediction the variance is $(0.15)^2=0.0225\;(\mathrm{mg/L})^2$, and the standard deviation is $0.15$ mg/L; the prediction does not change these values. (2) $\mathrm{SSE}=0.10^2+(-0.10)^2+0.20^2=0.0600\;(\mathrm{mg/L})^2$. Thus $\widehat{\sigma}_{\mathrm{add}}^2=0.0600/3=0.0200\;(\mathrm{mg/L})^2$, and $\widehat{\sigma}_{\mathrm{add}}=\sqrt{0.0200}=0.141$ mg/L. (3) $0.10/0.50=0.20$, or 20%. (4) A normal additive model assigns positive probability below zero, while concentration values may be nonnegative and observations near the assay limit may be censored rather than observed as uncensored numerical values.

Question	Additive-error implication	Potential warning
How does absolute spread change with prediction?	It remains \(\sigma_{\mathrm{add}}\) in response units.	Spread clearly rises or falls across predictions.
How does relative spread change?	It is larger when the prediction is smaller.	A constant relative spread appears more plausible than constant absolute spread.
Can a modeled observation be negative?	Yes, under the normal distribution.	Negative values are impossible, or near-limit data are censored.
Are residuals independent?	Conditional residual covariance between distinct observations is zero.	Residuals show within-subject dependence or shared assay effects.