Why Prespecify These Comparisons?
A model can describe unexplained variability in more than one way. Differences among individuals may be represented by interindividual random effects; changes for the same individual across occasions may call for interoccasion random effects; and observation-level deviations may be represented by residual error. These components answer different scientific questions. Comparing them without a plan can encourage choices driven by whichever model happens to fit best, even when that model is poorly supported by the design or data.
Prespecification makes the comparison interpretable. It records which alternatives are plausible, what changes between them, what assumptions are made, and what evidence will be summarized. The aim is not to guarantee that one candidate will win. It is to make the eventual analysis traceable and to ensure that a failed or unsupported comparison is documented rather than quietly replaced with a new one.
Define the Variability Components
Let \(i\) index individuals, \(j\) index occasions within an individual, and \(k\) index observations within an occasion. A typical population model predicts an observation through a function \(f_{ijk}\), whose value may depend on dose, time, and model parameters. The random-effects structure determines how individual- and occasion-specific parameters vary around their population values.
For example, a positive parameter such as clearance may be modeled on a log scale. With IIV alone, one common specification is \(CL_i=CL_{\mathrm{pop}}\exp(\eta_i)\), where \(\eta_i\sim N(0,\omega^2)\). With both IIV and IOV, a possible specification is \(CL_{ij}=CL_{\mathrm{pop}}\exp(\eta_i+\kappa_{ij})\), where \(\kappa_{ij}\) is an occasion-specific deviation. The interpretation of \(\omega^2\) and the IOV variance depends on the parameterization; variances are on the log scale in these examples.
Such equations do not automatically settle the dependence assumptions. A plan should say whether the random effects are independent, whether different random effects may be correlated, and whether occasion deviations are independent across occasions. For instance, a simple IOV structure may assume \(\kappa_{ij}\sim N(0,\pi^2)\), independently across occasions and independently of \(\eta_i\). That is a modeling choice, not a consequence of the marginal normal distributions. The data structure must also support the intended distinction: if individuals contribute only one occasion, separate IIV and IOV contributions may not be identifiable from those data alone.
Residual error also needs a clearly stated distribution and dependence assumption. An additive model might specify \(Y_{ijk}\mid\eta_i,\kappa_{ij}\sim N(f_{ijk},\sigma_a^2)\). A proportional model might specify \(Y_{ijk}=f_{ijk}(1+\epsilon_{ijk})\), with \(\epsilon_{ijk}\sim N(0,\sigma_p^2)\), giving conditional variance \(f_{ijk}^2\sigma_p^2\). A combined model can be written as \(Y_{ijk}=f_{ijk}+\epsilon_{a,ijk}+f_{ijk}\epsilon_{p,ijk}\); if the two error components are independent, centered, and have variances \(\sigma_a^2\) and \(\sigma_p^2\), its conditional variance is \(\sigma_a^2+f_{ijk}^2\sigma_p^2\).
Crucially, the stated marginal distribution or variance for each residual does not establish independence between observations. If the residual model assumes conditional independence, state it separately: conditional on the random effects and model predictions, residuals at distinct observations are independent. If serial, within-occasion, or other residual correlation is scientifically plausible, specify a joint distribution or correlation structure instead. Do not infer independence from a variance formula alone.
What the Plan Should Fix in Advance
A useful comparison set changes one prespecified feature at a time where feasible. It lists a reference model, candidate alternatives, the scientific rationale for each, and the conditions under which a candidate can be estimated. It also states how the same observations, structural model, estimation settings, and objective function will be used across candidates. Otherwise, an apparent improvement may reflect multiple simultaneous changes rather than the random-effects or residual-error choice of interest.
| Comparison | Scientific question | Prespecify |
|---|---|---|
| IIV structure | Which parameters plausibly differ among individuals? | Parameters with IIV, distributions, covariance assumptions, and minimum data/design support |
| IOV structure | Could a parameter vary within individuals across defined occasions? | Occasion definition, eligible parameters, dependence assumptions, and required repeated occasions |
| Residual-error structure | Does observation variability change with the prediction or measurement scale? | Additive, proportional, or combined form; units; and residual dependence assumptions |
The alternatives should be scientifically plausible, not merely mathematically constructible. IIV in clearance may be plausible where individuals differ in physiology or disease status; IOV may be plausible where the same individual is measured across treatment periods with potential within-person changes. A separate IOV term is less persuasive if “occasion” has no consistent scientific definition, or if there are too few repeated occasions to distinguish it from IIV. Residual-error choices should reflect the measurement process and scale. A proportional component may be plausible when the absolute spread increases with the predicted concentration; an additive component may be relevant near zero or where assay error has an approximately fixed absolute scale.
A Justification for Comparing Nested Variance Components
A common comparison adds a random effect whose variance is constrained to be nonnegative. The simpler model is obtained when that variance equals zero. This is a boundary case: zero is at the edge, not in the interior, of the allowed parameter space. Under suitable regularity conditions, a likelihood-ratio statistic for one such variance component has an asymptotic reference distribution that is a 50:50 mixture of a point mass at zero and a chi-square distribution with one degree of freedom. The result is not universal: multiple components, nuisance parameters that are unidentified under the null, small samples, or other irregularities can invalidate this approximation.
Here \(\ell\) denotes the maximized log likelihood under each model, and \(D\) is nonnegative when the larger model has at least as high a maximized likelihood. The mixture result follows because, under the null, the estimated variance is at the boundary with asymptotic probability one half; on that part of the limiting distribution the statistic is zero. On the remaining part, it follows the usual one-degree-of-freedom chi-square limit. The plan should name the intended comparison method and its assumptions, rather than applying an ordinary one-degree-of-freedom chi-square test automatically to every added random effect.
A likelihood-ratio comparison is not the only possible approach. A plan may instead use a prespecified information criterion, predictive evaluation, diagnostic evidence, or a combination. These approaches answer somewhat different questions and have their own limitations. Whichever approach is chosen, it should be applied consistently, and numerical fit improvement should be interpreted alongside parameter precision, diagnostics, scientific plausibility, and successful estimation.
Worked Example: Planning an IIV Comparison for Clearance
Suppose a hypothetical study includes repeated concentration measurements from multiple individuals. Before examining fit results, the team prespecifies a comparison between a model with fixed population clearance and one with lognormal IIV in clearance. The biological rationale is that individuals may differ in clearance. Other random effects and the residual model are held constant. The plan requires successful estimation, an uncertainty summary for the variance, and diagnostic review; the IIV model is not retained solely because its objective function is lower.
For illustration, suppose the fitted objective is \(-2\ell=482.0\) for the simpler model and \(474.8\) for the IIV model. The likelihood-ratio statistic is \(D=482.0-474.8=7.2\). If the one-component boundary-mixture approximation is appropriate, its tail probability is approximately \(\tfrac12P(\chi^2_1\ge7.2)\approx0.0036\). This calculation quantifies the evidence under that reference distribution; it does not establish that the model is scientifically correct or that the variance is precisely estimated.
Suppose the estimated log-scale variance is \(\hat{\omega}^2=0.160\). Its standard deviation on the log scale is \(\sqrt{0.160}=0.400\). For this lognormal parameterization, the corresponding coefficient of variation is \(\sqrt{\exp(0.160)-1}=0.417\), or about 41.7%. The plan should make clear that this is a model-based summary of between-individual variation in clearance, not a 41.7% standard deviation in the original scale. It should also specify reporting the estimate’s uncertainty and checking whether the estimate is near a boundary or poorly determined.
Worked Example: Deciding Whether IOV Is Estimable and Plausible
Consider a hypothetical repeated-dose study in which each of 30 individuals has measurements during two treatment occasions. The plan proposes testing IOV in clearance only if occasion is defined consistently and both occasions have sufficient observations across individuals. It specifies \(\log(CL_{ij})=\log(CL_{\mathrm{pop}})+\eta_i+\kappa_{ij}\), with independent centered normal IIV and occasion effects. The purpose of this structure is to distinguish stable between-person differences from within-person occasion deviations.
Suppose the objective values are 310.0 without IOV and 306.9 with IOV. Then \(D=310.0-306.9=3.1\). Under the same one-variance-component mixture approximation, the tail probability is approximately \(\tfrac12P(\chi^2_1\ge3.1)\approx0.0392\). This is an illustrative calculation conditional on the approximation’s assumptions; the result alone does not show that IOV is well estimated.
If the estimated IOV variance is \(\hat{\pi}^2=0.0900\), the log-scale standard deviation is \(\sqrt{0.0900}=0.300\). Its corresponding lognormal coefficient of variation is \(\sqrt{\exp(0.0900)-1}=0.307\), or 30.7%. The planned summary should also show uncertainty and diagnostics. If the estimate is unstable, if occasion effects are confounded with a systematic occasion difference, or if the fitting procedure fails, the plan should require those limitations to be recorded rather than treating the numerical comparison as decisive.
Worked Example: Prespecifying Residual-Error Alternatives
A hypothetical concentration analysis uses predictions in mg/L. The plan compares additive error, proportional error, and a combined model, keeping the structural and random-effects components fixed. The rationale is that absolute measurement error may matter at low concentrations while the spread may increase with concentration. In the combined model, suppose the planned variance is \(\sigma_a^2+f^2\sigma_p^2\), with \(\sigma_a=0.200\) mg/L and \(\sigma_p=0.100\).
At a prediction of \(f=2.00\) mg/L, the combined conditional variance is \(0.200^2+(2.00)^2(0.100)^2=0.0400+0.0400=0.0800\;(\mathrm{mg/L})^2\), so the conditional standard deviation is \(\sqrt{0.0800}=0.283\) mg/L. At \(f=10.0\) mg/L, the variance is \(0.0400+(10.0)^2(0.0100)=1.040\;(\mathrm{mg/L})^2\), and the standard deviation is \(1.02\) mg/L. Thus the planned model allows the absolute spread to change with the prediction; it does not assume that residuals are independent unless that is separately stated.
For an illustrative fit summary, suppose the objective values are 188.0 for additive, 176.6 for proportional, and 171.9 for combined error. The combined model has the lowest objective among these candidates, with a difference of \(176.6-171.9=4.7\) relative to proportional error. That difference alone is not a prespecified decision rule, especially because the candidate structures differ in parameter count and may have boundary issues. The plan should say how fit evidence will be balanced with residual diagnostics, parameter uncertainty, and whether the model can be estimated reliably.
Diagnostics, Failure Modes, and Alternatives
A prespecified comparison is useful only if the analysis can reveal when its assumptions or estimation have failed. Plan to examine whether variance estimates are at or near zero, whether standard errors are very large or unavailable, whether estimates change markedly with starting values, and whether the model converges to the same solution on repeated attempts. A nominally successful run does not ensure that a variance component is identifiable. For IOV in particular, examine whether the design includes within-individual repetition across the defined occasions and whether occasion patterns could instead reflect a systematic fixed effect.
Residual-error diagnostics should be considered across the prediction range and, where relevant, by time, occasion, and measurement scale. A proportional model can behave poorly near zero, and a normal proportional-error formulation can assign positive probability to negative observations. A combined model may be more flexible but can be difficult to estimate when the data do not contain enough information to separate its components. Correlated observations require a joint residual model or a defensible correlation structure; fitting independent residuals simply because each marginal variance looks reasonable can misrepresent uncertainty and patterns in the data.
The principal alternatives have different strengths. Likelihood-ratio comparisons are natural for nested models but need valid reference distributions and successful likelihood maximization. Information criteria can compare models with different parameter counts, but their ranking depends on the criterion and does not prove that the selected model is scientifically adequate. Predictive evaluation can emphasize performance for the intended prediction task, but depends on how prediction is defined and validated. Diagnostic plots and scientific reasoning help identify inadequacy that a single statistic may miss, but should be described with explicit criteria rather than used as an unrecorded reason to favor a preferred model.
The plan should therefore define in advance what will be summarized for each candidate: the model specification, variance estimates and units or scale, uncertainty, objective or chosen comparison statistic, estimation status, and relevant diagnostics. It should also state what happens when a comparison is not estimable or assumptions for the planned test do not hold. A transparent conclusion may be that the data do not support distinguishing two structures; that is more informative than presenting an unstable choice as certain.
Common Mistakes
- Treating IIV and IOV as interchangeable. IIV describes between-person differences, whereas IOV describes within-person variation across defined occasions. The design must contain repeated occasions to support their separation.
- Assuming residual independence from marginal variances. A variance formula specifies each residual’s spread, not the dependence between residuals. State conditional independence as an additional assumption or specify a joint correlation model.
- Adding every possible random effect. A candidate needs a scientific rationale and sufficient design information; extra variance terms can be weakly identified or redundant.
- Using an ordinary chi-square reference automatically. A variance-component null often lies on a boundary. Prespecify an appropriate method and verify its conditions.
- Choosing the lowest objective without qualification. Fit improvement is only one part of the assessment; estimation stability, precision, diagnostics, and plausibility also matter.
Check Your Understanding
Answer each question using the distinctions and planning principles in this tutorial.
- A study has one occasion per individual. What limitation does this impose on separating IIV from IOV?
- Does specifying \(Y_{ijk}\mid\eta_i\sim N(f_{ijk},\sigma^2)\) imply that residuals at different observations are independent? What must be added if independence is intended?
- For a proportional-error model with \(f=4.00\) mg/L and \(\sigma_p=0.150\), calculate the conditional residual standard deviation.
- Why can a standard one-degree-of-freedom likelihood-ratio reference be inappropriate when testing whether a random-effect variance is zero?
- Name three elements, beyond a fit statistic, that a prespecified comparison summary should include.
Solutions. (1) With only one occasion per individual, within-individual occasion variation is not observed, so IIV and IOV generally cannot be separated from these data alone. (2) No. The marginal normal distribution specifies a residual’s conditional distribution, not the joint dependence across observations. State conditional independence separately, or specify a joint residual distribution or correlation structure. (3) The variance is \(f^2\sigma_p^2=(4.00)^2(0.150)^2=16(0.0225)=0.360\;(\mathrm{mg/L})^2\), so the standard deviation is \(\sqrt{0.360}=0.600\) mg/L. (4) The null value zero is on the boundary of the nonnegative variance parameter space, which can change the limiting reference distribution; the appropriate method depends on the comparison and its assumptions. (5) Examples include variance estimates and their scale, uncertainty, estimation or convergence status, diagnostic evidence, and scientific plausibility.