Why the Order of Decisions Matters
An NLME model is assembled from linked choices. Its structural model describes the expected response over time; its random-effects structure describes between-subject variation in model parameters; its residual-error model describes remaining observation-level variation; and its covariate model describes systematic relationships between subject characteristics and parameters. A choice at one layer can change what the other layers appear to explain.
For example, a structural model that misses a delayed response may leave a time-patterned residual. Adding a random effect or a covariate can sometimes improve the numerical fit while obscuring that structural problem. Conversely, an overly restrictive random-effects model can make a covariate appear to explain variability that is actually between subjects. The earlier material on prospective NLME analysis plans establishes the value of planning decisions in advance. Here the focus is how to order those decisions and when to revisit them.
A general representation is
Here \(Y_{ij}\) is subject \(i\)'s observation at time \(t_{ij}\), \(f\) is the structural model, \(\theta_i\) is that subject's parameter vector, \(\theta\) denotes typical parameters, \(\eta_i\) denotes random effects, \(x_i\) denotes measured covariates, and \(\beta\) denotes covariate-effect parameters. A common working assumption is \(\eta_i\sim N(0,\Omega)\), with \(\Omega\) the random-effects covariance matrix. Conditional on the subject effects and covariates, residuals have mean zero and a specified variance structure \(R_{ij}\); observations are conditionally independent unless the model explicitly specifies otherwise. These assumptions must be assessed against the data and scientific context, not treated as automatic truths.
A Defensible Sequence, With Planned Return Paths
A useful default sequence is: establish a plausible structural model; evaluate the random-effects structure; evaluate residual error; then evaluate covariates. This order is a working strategy, not a claim that each layer can be settled independently. A baseline residual model is needed from the outset to fit and compare structural candidates. It should be scientifically reasonable and adequate for the comparisons being made, then examined more deliberately after the structure and variability have been assessed.
Use the same analysis population, observations, response scale, and likelihood convention when comparing candidates. State what scientific features the candidate models are intended to represent.
Compare plausible mean-response structures using an initial, defensible variability and residual specification. Inspect predictions and residual patterns as well as fit summaries.
With a credible structural model in place, evaluate which parameters need random effects and whether estimated variability and covariance are supported by the data.
Check whether the residual variance model describes the remaining observation-level scatter across the prediction range and relevant times.
Test scientifically motivated relationships against a stable base model. Recheck estimates and diagnostics after a covariate is introduced.
Systematic residual patterns, unstable estimates, or implausible parameter behavior can indicate that the structural or variability choices need revision.
This sequence separates questions that are easy to confound. In particular, covariates are usually easier to interpret after the model already represents major structural features and between-subject variation. Nevertheless, if the initial residual model is clearly inadequate, it should be improved before relying on comparisons that depend on it. The goal is not to postpone every error-model decision; it is to avoid treating a convenient initial assumption as permanently settled.
Worked Comparisons and What They Show
Worked Example: A Structural Ranking Changes After Variability Is Added
Suppose two scientifically plausible structural candidates, A and B, are fitted to the same hypothetical data. At the initial stage, both use the same baseline residual model. For illustration, candidate A has \(-2\log L=210\) with \(k=9\) estimated parameters, while B has \(-2\log L=206\) with \(k=10\). Use the Akaike information criterion, \(AIC=-2\log L+2k\), for this comparison.
For A, \(AIC=210+2(9)=228\). For B, \(AIC=206+2(10)=226\). At this stage B has the lower AIC by 2 units, so it is provisionally favored by this criterion, not established as the uniquely correct structure.
After investigating between-subject variability, suppose a supported random effect is included in each candidate. The refitted values are A: \(-2\log L=190\), \(k=12\); and B: \(-2\log L=193\), \(k=13\). Then A has \(AIC=190+2(12)=214\), while B has \(AIC=193+2(13)=219\). The ranking now favors A by 5 AIC units.
These numbers are hypothetical, and each AIC comparison assumes the same observations and likelihood definition within that comparison. The reversal illustrates why an initial structural ranking can depend on the provisional variability specification. It does not prove that adding any random effect is appropriate: the added effect must also be estimable, scientifically interpretable, and consistent with diagnostics.
Worked Example: Evaluating a Random-Effects Extension
Consider two fits of the same hypothetical structural and residual models, using the same observations and likelihood. Model R0 has one random effect, \(-2\log L=480\), and \(k=15\). Model R1 adds a random effect on a second parameter, giving \(-2\log L=468\) and \(k=16\). The AIC values are \(480+2(15)=510\) and \(468+2(16)=500\), respectively.
R1 has an AIC 10 units lower. The improvement in fit is large enough to outweigh the one-parameter penalty under this criterion. Before retaining R1, inspect whether the new variance estimate is stable and meaningful, whether estimates and uncertainty for other parameters remain plausible, and whether the fit is supported by subject-level patterns. If the added variance is near zero, poorly identified, or associated with estimation problems, the AIC difference alone is not sufficient justification.
Worked Example: A Weight Effect Depends on the Base Model
Suppose clearance is modeled with a weight covariate as \(CL_i=\theta_{CL}(WT_i/70)^\beta\exp(\eta_{CL,i})\), where \(WT_i\) is weight in kilograms and \(\theta_{CL}\) is typical clearance at 70 kg. In a hypothetical initial base model, adding the one-parameter weight effect changes \(-2\log L\) from 300 to 294 and increases \(k\) from 11 to 12. The base AIC is \(300+2(11)=322\); the covariate AIC is \(294+2(12)=318\). The covariate model is favored by 4 AIC units in this comparison.
Suppose its fitted \(\beta\) is 0.30. At 84 kg, the weight multiplier is \((84/70)^{0.30}=1.2^{0.30}=\exp(0.30\log(1.2))\approx\exp(0.0547)\approx1.056\). Thus, for a typical clearance of 10 L/h at 70 kg, the covariate model predicts about \(10(1.056)=10.56\) L/h at 84 kg, before accounting for the individual random effect.
After the random-effects and residual models are better supported, suppose the base fit has \(-2\log L=260\), \(k=15\), while the weight model has \(-2\log L=258.5\), \(k=16\). Their AIC values are 290 and 290.5, respectively. The covariate no longer has a lower AIC. If the updated estimate is \(\beta=0.10\), the corresponding 84 kg prediction is \(10(1.2^{0.10})\approx10(1.018)=10.18\) L/h. The changed fit and estimate show why a covariate conclusion should be conditional on a credible base model. They do not establish that weight has no relationship with clearance.
Diagnostics, Alternatives, and Limitations
At each stage, ask whether the model is explaining the kind of variation it was introduced to explain. Structural inadequacy may appear as residual trends over time or predictions that systematically miss a response phase. An inadequate random-effects structure may leave persistent subject-level deviations or produce unstable estimates of variability. An inadequate residual model may show changing scatter across prediction levels. A covariate model deserves scrutiny if its effect changes greatly under modest, scientifically reasonable changes to the base model.
These patterns are diagnostic clues, not automatic instructions to add parameters. A visible trend might have multiple explanations; the proposed change should match the pattern and have a scientific rationale. Convergence alone does not demonstrate adequacy, and a favorable fit criterion does not guarantee reliable individual parameter estimates or useful predictions.
| Approach | Strength | Main risk |
|---|---|---|
| Strict one-pass sequence | Separates decisions and can reduce opportunistic changes. | May preserve an early misspecification even when later diagnostics expose it. |
| Change several layers at once | Can improve fit quickly when the initial model is poor. | Makes it difficult to identify which change mattered or to interpret estimates. |
| Ordered evaluation with targeted returns | Provides interpretable comparisons while allowing evidence-based revision. | Requires judgment about which diagnostic pattern warrants revisiting an earlier choice. |
The third approach is generally the most defensible. Its limitation is that no universal ordering removes dependence between model components. Sparse sampling, limited numbers of subjects, or weak variation in a covariate can leave several plausible explanations difficult to distinguish. Comparisons should therefore be interpreted in light of the design and the model's intended use. Keep the comparison conditions stable, make one targeted change where possible, and revisit earlier layers only when evidence gives a reason to do so.
Common Pitfall
Check Your Understanding
Use the sequence and comparison principles above. The solutions follow the questions.
- Why is a provisional residual-error model still needed while comparing structural candidates?
- A model's \(-2\log L\) improves by 3 when one parameter is added. What is the change in AIC, and which model has the lower AIC?
- What conclusion is justified if a covariate is favored under an initial model but not under a better-supported random-effects and residual specification?
- Name one diagnostic pattern that could justify returning to the structural model after evaluating random effects.
Solutions. (1) The likelihood fit requires a residual specification; structural comparisons are conditional on that specification, so it must be plausible and consistently applied. (2) The change is \(\Delta AIC=-3+2=-1\), so the model with the added parameter has the lower AIC by 1 unit, a modest criterion-based preference rather than proof. (3) The covariate conclusion depends on the base model and is not robust to that change; neither result alone establishes the presence or absence of a covariate relationship. (4) A systematic residual trend over time or a consistently missed response phase can indicate structural inadequacy and support a targeted structural review.