Why Prespecify Model-Development Decisions?
A population PK analysis plan is more useful when it says not only what question the analysis addresses, but also how plausible model-development choices will be handled. If a decision is left vague until results are visible, it is easy to choose the option that appears most favorable after the fact. Prespecification makes the reasoning reviewable: a reader can tell which choices followed a planned rule and which were made in response to the observed results.
The earlier material on writing a population PK analysis plan establishes the broader role of that plan. Here the focus is narrower: recording decisions about the structural model, random effects, residual error, covariates, and comparisons among models. Prespecification does not require predicting every outcome or forbidding scientific judgment. It requires stating the intended options, decision criteria, and limits of those criteria before using the analysis results to choose among them.
A plan should be concrete enough that an independent analyst could follow its rules. “Choose the best model” is not operational: it does not say what models are eligible, what “best” means, or what happens if estimation fails. In contrast, a plan can specify a primary candidate, a limited set of alternatives, the order or conditions for evaluating them, the criteria for comparison, and how unresolved conflicts will be handled.
What to Record Before Model Fitting
The plan need not teach the reader how to build every possible PK model. It should identify the decisions that matter for this analysis and state how they will be made. The level of detail should prevent an analyst from adding, removing, or redefining options without making that change visible.
| Decision area | What the plan should make explicit |
|---|---|
| Structural model | The primary structural candidate and any planned alternatives; the features that distinguish them; and the criteria for considering an alternative. |
| Random effects | Which parameters may have between-subject variability, the planned distribution and covariance structure, and any prespecified simplification rule. |
| Residual error | The primary residual-error form, the scale on which it applies, and any alternative form or comparison rule. |
| Covariates | The covariates to evaluate, their coding and scaling, the parameterization, and the criteria for retaining or interpreting an effect. |
| Model comparison | The eligible comparison method, its assumptions, thresholds if used, diagnostic requirements, and a rule for inconclusive or conflicting evidence. |
A prespecified candidate set is not a guarantee that every candidate can be estimated or that one will be clearly preferred. A model may fail to converge, produce implausible estimates, or be poorly supported by the information in the data. The plan should say in advance how such outcomes affect the decision. It should not convert a failed fit into evidence for a preferred alternative.
Worked Example: Turning Broad Intent Into a Structural Plan
Suppose a hypothetical analysis concerns an intravenously administered drug. The team expects to begin with one-compartment disposition and wants a planned way to consider a more flexible alternative. “Try different structures and choose the best fit” would leave the candidate set and decision rule undefined.
A more operational plan could state: “The primary structural candidate will use one-compartment disposition. A two-compartment candidate will be evaluated as the sole planned structural alternative if the primary candidate converges but displays a prespecified pattern of systematic concentration-time residuals. The alternative will be considered only if it also has acceptable parameter precision and no material diagnostic deterioration. If the primary candidate fails estimation, the analysis will report the failure and evaluate the alternative as a contingency, not as a successful result of the planned comparison.”
This wording names the primary model, limits the planned alternative to one candidate, and distinguishes a planned diagnostic-triggered comparison from a contingency after fitting failure. Before fitting, the candidate count is \(1+1=2\): one primary candidate and one planned alternative. That count does not imply both models will be estimable or that the alternative must be selected. The diagnostic pattern and practical criteria should be defined sufficiently clearly in the actual plan for consistent application.
The example does not establish that either structural choice is appropriate for a particular drug. It shows how to record a decision pathway without presenting a model selected after examining results as though it had been the planned primary model.
Random Effects and Residual Error
For random effects, the plan should identify which individual-level parameters may vary between subjects and whether the initial covariance structure is diagonal or permits correlations. It should also define any planned simplification, such as removing a correlation that cannot be estimated reliably. A statement such as “include random effects as needed” leaves the decision to an unspecified, potentially outcome-dependent judgment.
The number of random-effect parameters can change with the covariance specification. For two random effects, a diagonal covariance matrix has two variance parameters. A fully unstructured covariance matrix has two variances and one covariance, so it has \(3-2=1\) additional parameter. This difference matters for estimation and comparison, but it does not by itself establish that the more complex structure is appropriate. The data must support estimating its additional parameter, and the plan should state how instability or poor precision will be handled.
For residual error, name the planned form and the concentration scale to which it applies. If alternatives are allowed, specify them rather than adding forms opportunistically after inspecting residual plots. A prespecified order may be useful, but it should not imply that a more complex error model automatically improves the scientific model: a change in residual error can alter the apparent fit of structural or covariate components.
Worked Example: Prespecifying a Covariate Parameterization
Consider a hypothetical plan to evaluate body weight as a continuous covariate on clearance, \(CL_i\), for subject \(i\). The plan specifies weight in kilograms, scales it by 70 kg, and evaluates a power relationship:
Here \(WT_i\) is subject \(i\)'s body weight, \(\theta_{CL}\) is the typical clearance at 70 kg when the random effect is zero, \(\theta_{WT}\) is the weight exponent, and \(\eta_{CL,i}\) is the individual random effect on clearance. A complete plan would also state whether this is the sole planned weight relationship, which estimation and diagnostic criteria apply, and whether weight will be evaluated on any other parameter.
For illustration only, suppose \(\theta_{CL}=6.00\) L/h, \(\theta_{WT}=0.75\), and \(WT_i=84\) kg, with \(\eta_{CL,i}=0\). Then:
This calculation shows what the chosen parameterization means at a specified weight; it is not evidence that the covariate should be retained. The plan makes the hypothesis testable and reproducible by fixing the covariate, scale, functional form, and target parameter before results are examined. Choosing a different weight transformation only after seeing which one gives the most favorable fit would be a post hoc choice.
Prespecifying Model-Comparison Rules
Comparison criteria should match the question and the eligible candidates. For nested models fitted to the same observations using the same likelihood, a likelihood-ratio comparison may be considered. Define the objective function as \(OFV=-2\log L\), where \(L\) is the maximized likelihood. For a reduced model and a larger nested model, the difference is:
Under the null hypothesis that the added parameters are unnecessary, and under regularity conditions, \(\Delta OFV\) is approximately distributed as chi-square with degrees of freedom equal to the number of added parameters. The models must be nested, use the same data and likelihood basis, and be estimated adequately. The approximation can fail in settings such as variance components on a boundary, so a plan should not apply a routine chi-square threshold without considering whether its assumptions are reasonable.
Worked Example: Applying a Prespecified Likelihood-Ratio Rule
Suppose a hypothetical plan specifies a likelihood-ratio comparison at \(\alpha=0.05\) for two nested models that differ by one fixed covariate-effect parameter. Both models use the same observations and likelihood, and both fit successfully. The reduced model has \(OFV=412.6\); the larger model has \(OFV=405.4\).
The calculated difference is:
With one added parameter, \(df=1\). The 5% upper-tail critical value is approximately 3.841, and \(7.2>3.841\). The corresponding chi-square upper-tail probability is approximately \(0.0073\). Under the stated assumptions, this comparison meets the prespecified statistical criterion for improved fit from the added parameter.
That result alone does not establish that the covariate effect is biologically important, well estimated, or suitable for the final model. The plan should also require acceptable estimation and diagnostic evidence. If the models were not nested or the likelihood-ratio approximation were not justified, this rule would not be a valid basis for the same conclusion.
For candidates that are not nested, or when a likelihood-ratio test is not appropriate, an information criterion such as AIC may be specified. With \(k\) estimated parameters and \(OFV=-2\log L\), \(AIC=OFV+2k\), up to a constant common to the candidates. Lower AIC is preferred within the comparison. AIC is a relative criterion, not a hypothesis test: it does not provide a p-value or prove that the lower-AIC model is true. Candidates must be fitted to the same observations with a comparable likelihood for the comparison to be meaningful.
Worked Example: Comparing Candidates With AIC
In a separate hypothetical comparison, two adequately fitted candidates use the same observations and comparable likelihoods. Candidate A has \(OFV=405.4\) and \(k=8\); candidate B has \(OFV=400.0\) and \(k=10\). The plan states that AIC will be used for this comparison.
For candidate A, \(AIC=405.4+2(8)=421.4\). For candidate B, \(AIC=400.0+2(10)=420.0\). The difference is \(421.4-420.0=1.4\), so candidate B has the lower AIC by 1.4 and is preferred by this prespecified criterion.
The result is a relative ranking under the chosen criterion, not proof that candidate B is correct. The plan should state how AIC will be weighed alongside diagnostics, parameter precision, and scientific plausibility, particularly when the criterion gives only a small separation.
Diagnostics, Alternatives, and Limits
A decision rule is useful only if its assumptions and prerequisites are checked. Before applying a comparison criterion, confirm that candidates were estimated adequately, that the parameter count and model relationship are correctly identified, and that the same data and likelihood basis were used where required. Inspect the planned diagnostics for systematic residual patterns, implausible estimates, excessive uncertainty, or signs that parameters cannot be distinguished from one another.
Common approaches have different strengths. A fixed candidate sequence is transparent and limits unplanned exploration, but may not cover every plausible model. A likelihood-ratio rule gives a formal comparison for suitable nested models, but its reference distribution depends on assumptions. AIC can compare nonnested candidates under comparable likelihoods, but supplies a relative ranking rather than a significance test. A broad data-driven search may reveal unexpected patterns, but increases the risk of selecting a model because it happened to fit the observed data well. A plan can allow exploratory analyses while clearly separating their results from decisions made under prespecified rules.
Prespecification also has limits. It cannot ensure that the selected candidates are biologically adequate, make sparse data informative, or guarantee stable estimation. Nor should it force a model to be retained when diagnostics indicate a serious problem. When a planned rule cannot be applied or evidence conflicts, the appropriate response is to explain the limitation and label any new choice accurately, rather than retroactively calling it prespecified.
Common Pitfall
Correction: Prespecifying a general intention is not the same as prespecifying the covariate candidates, parameterization, and selection rule. If transformations or candidates were added after results were reviewed, those choices are post hoc, even if the broader covariate analysis was planned.
Another warning sign is a rule with undefined terms such as “acceptable fit,” “important improvement,” or “if needed.” Such wording may be unavoidable for some scientific judgments, but it should be accompanied by operational criteria, named diagnostics, or an explicit statement of who will make the judgment and what evidence will be considered. Otherwise, different analysts can apply the same plan differently.
Name the structural candidates, random-effect and residual-error options, covariates, and comparison methods that are genuinely intended.
Specify parameterizations, thresholds where justified, diagnostics, model prerequisites, and how to handle failed or inconclusive comparisons.
Record when an analysis goes beyond the stated candidate set or rule; do not relabel a result-driven choice as prespecified.
Check Your Understanding
Answer each question using the definitions and decision principles above.
- A plan says, “We will choose the best residual-error model.” What key information is missing?
- Two nested models differ by one parameter and have objective functions 250.0 and 245.9. Calculate \(\Delta OFV\) and compare it with the 5% chi-square critical value for one degree of freedom. What additional conditions matter?
- A team planned a linear weight effect but, after seeing the results, instead uses a power effect because it fits better. Was the power effect prespecified? Explain.
- For two random effects, how many covariance parameters are in a diagonal structure and in a fully unstructured structure? How many additional parameters does the latter have?
Solutions. (1) The plan needs to identify the candidate error forms, the scale and comparison criteria, and how diagnostics or estimation failure affect the choice. (2) \(\Delta OFV=250.0-245.9=4.1\), which exceeds the 1-df 5% critical value of approximately 3.841. This supports the larger model by that rule only if the models are nested, adequately estimated, use the same data and likelihood, and the chi-square approximation is justified. (3) No. The planned linear effect was prespecified; adopting the power effect after reviewing results is a post hoc choice. (4) The diagonal structure has two variances; the unstructured structure has two variances and one covariance, for three parameters total and one additional parameter.