Writing a Prospective NLME Analysis Plan

Why Plan Before Fitting?

An NLME analysis involves a series of choices: which records to analyze, how to represent the population and individual parameters, what evidence will be used to judge model adequacy, and how uncertainty will be described. Some choices can be stated precisely before model fitting; others may require a justified adaptation when the data reveal an unanticipated problem. A prospective analysis plan makes the first group explicit and defines how the second group will be handled.

The plan is not a promise that one model will fit, nor a way to eliminate scientific judgment. It is a guide for applying that judgment consistently. It should make clear what question the analysis addresses, what data and methods are intended, what evidence will support decisions, and how departures from the plan will be recorded. The earlier material on defining the decision question and analysis population and creating a traceable NLME analysis dataset provides important inputs: the plan should use those definitions rather than silently changing them.

What the Plan Should Make Explicit

Begin with the objective, the population, and the intended use of the results. State the analysis population and the rules for membership, including any relevant exclusions. Describe the data source and the planned analysis-data version or cutoff, the response and time variables, units, dosing information, covariates, and identifiers needed to connect records to individuals. Clarify which outputs answer the decision question—for example, population parameter estimates, between-individual variability, individual predictions, or uncertainty summaries. Do not use a broad phrase such as “analyze the pharmacokinetic data” when a more specific definition is available.

Next, describe the planned modeling work at a level that is useful without pretending every choice can be known in advance. A general NLME model can be represented as an observation \(y_{ij}\) for individual \(i\) at observation \(j\), with a structural prediction depending on individual parameters and an observation-level residual:

Here \(t_{ij}\) denotes the relevant observation time, \(\theta_i\) the individual parameters, \(\theta_{\mathrm{pop}}\) population parameters, \(\eta_i\) individual random effects, and \(x_i\) covariates. This expression is a framework, not a complete model specification. The plan should identify the intended model-development scope and evaluation approach, while avoiding claims that the final model is known before the data are analyzed.

Decision criteria should be observable and interpretable. For each important decision, specify the evidence to be considered, any threshold or rule, and what happens if the criterion is not met. Thresholds are project-specific: no single fit statistic, convergence rule, or predictive check is a universal guarantee of model adequacy. Distinguish essential conditions from supporting evidence. For example, a model that fails to produce a valid estimate for a parameter needed for the stated objective should not be rescued by a favorable visual diagnostic elsewhere.

Data Handling and Decision Rules

Data handling rules should be written before fitting, not inferred from which records make a candidate model look better. State how missing values in essential fields will be identified; whether records with missing time, dose, or response can be used; how duplicated or inconsistent records will be investigated; and how below-quantification observations, if present, will be represented or otherwise handled. Specify whether implausible values trigger a data query, a documented correction, exclusion under a defined rule, or retention with a sensitivity analysis. “Remove outliers” is not an adequate rule: it does not define an outlier or explain how to distinguish a data error from an unusual but valid observation.

A useful plan also states how uncertainty in data interpretation will be handled. For instance, if a dose time is ambiguous, identify who or what source will be consulted and what happens if it cannot be resolved. Do not treat missingness as automatically harmless or assume that deleting incomplete records leaves the analysis unbiased. The consequences depend on why values are missing and on the analysis objective.

Separate model-fitting conditions from model-evaluation evidence. Conditions might include successful completion of the planned estimation procedure and availability of finite, interpretable estimates for parameters required by the objective. Evaluation evidence might include observed-versus-predicted patterns, residual patterns, predictive checks, and stability under reasonable changes. A single numerical score can conceal a serious local failure, so the plan should state which checks are complementary and how conflicting evidence will be discussed.

Worked Examples of Prospective Specifications

Why Decision Rules Need a Rationale

A concise justification makes a criterion less arbitrary and easier to interpret. Consider a decision that requires both an estimation condition \(C\) and a predictive-evaluation condition \(P\). If both are essential, the decision rule is their conjunction:

This form matters. If a model must have usable estimates and acceptable predictive behavior, a strong predictive check cannot compensate for a failed estimation condition, and vice versa. Replacing two essential conditions with an average score could hide that failure. In the plan, label criteria as essential gates or as supporting evidence and specify how discordant results will be interpreted. This does not turn the decision into a purely mechanical exercise; it makes the role of judgment visible.

The rationale should connect a threshold to the analysis purpose, data characteristics, or practical consequences. If a threshold is only a screening convention, say so. When no defensible numerical cutoff exists, prespecify the evidence to be reviewed and the principles for interpretation rather than inventing a precise number that implies unjustified certainty.

Reporting Choices, Alternatives, and Limitations

A prospective plan is different from a final analysis report. The plan states what is intended before relevant results are examined; the report states what was done and what was found. A plan may include anticipated alternatives and triggers for considering them, but an anticipated option is not automatically the final choice. Conversely, an unanticipated change does not become prespecified simply because it was later written into a dated document.

Compared with an entirely flexible exploratory workflow, a prospective plan reduces the opportunity to select criteria after seeing which results they favor. Compared with an inflexible rulebook, a plan with defined contingencies allows a reasoned response when data or computation reveal an unforeseen issue. The aim is neither to prohibit adaptation nor to disguise it: it is to distinguish planned decisions from decisions made in response to observed information.

The plan should say how both planned and changed procedures will be reported. At minimum, describe the procedure actually used, whether it matched the plan, the reason for any departure, when the departure occurred relative to examining relevant results, and its likely implications. Keep the original plan identifiable and date amendments; a revised plan should not obscure what was initially specified. Detailed model-decision records and computational provenance may be maintained in supporting materials, while the analysis report should provide enough information to understand the conclusions.

A plan cannot fix an unsuitable scientific model, poor data, or weak information in the study design. Prespecified criteria can also be too permissive, too restrictive, or poorly aligned with the decision question. The plan should therefore acknowledge important limitations, avoid treating a threshold as proof, and preserve room to explain evidence that the rules do not capture.

Common Pitfalls and Diagnostics

• Writing criteria after seeing the fit. This makes a result-guided choice appear prospective. Date the plan before examining the relevant outputs, and identify later amendments as amendments.
• Using vague instructions. “Check fit and remove bad records” is not reproducible. Name the diagnostics, define record-handling rules, and state how findings affect the next action.
• Using one threshold as a complete validation. A pass on a summary metric does not rule out systematic residual patterns, unstable estimates, or a misspecified model. Use complementary evidence and report limitations.
• Changing data rules without assessing impact. When a justified deviation affects records, describe the affected records and, where feasible, compare results under the planned and changed handling.
• Confusing a planned contingency with an actual decision. State whether the contingency was triggered and what was implemented; listing an option in advance does not mean it was used.