Separating Parameter Variability From Residual Error

Two Sources of Deviation, Two Places in the Model

A measured concentration can differ from the population prediction for two conceptually different reasons. The subject may have a clearance or volume that differs from the population-typical value, changing that subject’s predicted concentration profile. Or, even after using the subject-specific parameter, a particular measured concentration may lie above or below its model prediction. The first is between-subject parameter variability; the second is residual error.

The distinction is made by locating each source in the hierarchical model. A subject-level random effect changes an individual parameter and is shared across that subject’s observations. A residual error is attached to an observation, so a new observation can have a different residual even for the same subject and the same underlying parameter.

For this simple version, assume \(\eta_i\) has mean zero and variance \(\omega^2\), with independent values across subjects. Assume residuals have mean zero and variance \(\sigma^2\), are independent across observations, and are independent of the subject effects. A common illustrative specification is \(\eta_i\sim N(0,\omega^2)\) and \(\epsilon_{ij}\sim N(0,\sigma^2)\). The normality assumptions are useful for some likelihood calculations, but the conceptual distinction does not depend on normal distributions: it depends on where each term enters the model.

In the exponential parameter model, \(\theta_i\) is positive when \(\theta>0\), and \(\eta_i=0\) gives \(\theta_i=\theta\). The subject effect acts on the parameter scale through the exponential transformation. By contrast, additive \(\epsilon_{ij}\) acts on the observation scale in the equation above. These variances therefore refer to different levels and may have different units; they should not be treated as interchangeable measures of variability.

Worked Examples: Following Each Effect Through the Equations

Why Repeated Observations Help

The equation itself gives a useful way to understand the distinction. Conditional on a subject’s random effect, the model prediction is \(f(t_{ij},\theta_i)\). If residual errors have conditional mean zero, then

Thus, repeated observations from a subject inform the pattern predicted by that subject’s parameter, while individual deviations around the predictions are assigned to residual error under the model. A subject effect is shared across those observations; residual errors are not. This is why the number and timing of observations can matter: with very little information per subject, the data may not clearly distinguish a subject-specific parameter shift from observation noise.

For the simple repeated-measurement model \(y_{ij}=\theta+\eta_i+\epsilon_{ij}\), suppose in addition that \(\eta_i\) and \(\epsilon_{ij}\) are independent, mean zero, and have variances \(\omega^2\) and \(\sigma^2\). The variance of one observation across subjects and observations is

The equality follows because independent random variables have zero covariance, so \(\operatorname{Var}(\eta_i+\epsilon_{ij})=\operatorname{Var}(\eta_i)+\operatorname{Var}(\epsilon_{ij})\). This decomposition is for the stated additive model and assumptions; it is not a general recipe to add variance estimates from different model scales. In a nonlinear model, the relationship between parameter variation and observation variation depends on the structural function \(f\).

Assumptions, Diagnostics, and Common Confusions

The distinction is useful only when the model’s assumptions are plausible. In the simple specification, the subject effect is stable over the observations being modeled, residuals have conditional mean zero, and the residual-error form and dependence assumptions are adequate. For an additive residual model, errors are measured in the same units as \(y_{ij}\). Other residual specifications, such as proportional error, define the observation-level deviation differently; they do not turn an observation error into a subject parameter effect.

When reviewing a fitted model, ask whether deviations show a persistent subject-level pattern or are largely observation-specific. Compare observations with predictions that use the subject-specific parameter, and inspect residuals against time and predicted values. Repeated residuals of similar sign or a systematic time pattern can suggest that the structural model, covariate model, or residual specification is inadequate. Such a pattern is a reason to investigate, not proof that a particular random effect must be added. Conversely, a model that assigns nearly all discrepancies to residual error may fail to represent real differences in subject parameters.

A common mistake is to treat any measured value above the population-typical prediction as a positive subject random effect. That reasoning skips the distinction between the two levels. The correct sequence is to determine the subject parameter from the parameter model, calculate the structural prediction using that parameter, and then define the observation residual as observed minus predicted. A value can be above the population prediction because the subject parameter differs, because the observation residual is positive, or because both contribute.

Another limitation is that an estimated residual is not automatically a direct measurement of a physical error source. It is the discrepancy left after applying the chosen model. If the structural model omits a relevant process, the residual may contain systematic model misspecification as well as unexplained observation variation. Likewise, a fitted random-effect distribution is a model for population heterogeneity, not a guarantee that every individual parameter is known precisely.

Comparing the Two Terms

These are not competing explanations from which one must always choose only one. A model may include both, as in the equations above. The distinction is instead about how each source is represented and what pattern it is intended to explain.