Forward And Backward Covariate Model Building

Why Covariate Selection Needs a Defined Search

A list of plausible covariates is not yet a covariate model. Model building requires a rule for deciding which candidate relationships to include, and a precise description of what changes between the models being compared. Forward inclusion and backward deletion are two sequential strategies for doing this. They can organize a search, but neither strategy turns a data-driven choice into proof that a covariate has a biological or causal effect.

Here the target is a typical population PD parameter, such as a typical baseline response or drug-sensitivity parameter. A covariate relationship is a prespecified functional form linking a measured covariate to that parameter. For a positive parameter and a centered, scaled continuous covariate \(z_i\), one possible relationship is

where \(\theta_i\) is the parameter for individual \(i\), \(\theta_{\mathrm{typ}}\) is its typical value at \(z_i=0\), \(\beta\) is the covariate coefficient, and \(\eta_i\) represents between-subject random deviation. For a binary covariate, \(z_i\) can instead indicate membership in a specified category, with the comparison category as the reference. The candidate effect is absent when \(\beta=0\); including it adds \(\beta\) as a parameter to estimate. Other functional forms are possible, but the form and the parameter to which it applies must be chosen before the model comparison.

These procedures are most interpretable when models are fitted to the same observations with the same PD structural model, random-effects structure, residual model, and estimation settings. The candidate models should differ only in the relationship under evaluation. If one model uses different data, a different response model, or a materially different estimation setup, a change in fit cannot be attributed cleanly to the covariate change.

Objective-Function Changes and the Search Logic

Suppose models are fitted by maximum likelihood and the objective function is \(\mathrm{OFV}=-2\log L\), where \(L\) is the maximized likelihood. A smaller OFV indicates a better likelihood fit. For an added covariate, define the decrease in OFV as the base-model OFV minus the expanded-model OFV. For a deletion, define the increase as the reduced-model OFV minus the fuller-model OFV. Those are different comparisons, so report the direction and the models explicitly.

For nested models fitted successfully to the same data by the same likelihood method, the larger model contains all parameter settings available to the smaller model, as well as the option of setting the added coefficient to zero. Therefore its best attainable likelihood cannot be worse. Equivalently, its minimum attainable OFV cannot be larger. This follows because optimizing over a parameter space that contains another space cannot yield a worse optimum. In practice, numerical optimization can fail to find that optimum, so a violation of this property should prompt investigation rather than automatic interpretation.

This property explains why adding a term will generally improve or leave unchanged the fit, even when the term is not worth retaining. A search therefore needs a prespecified decision rule, not merely the rule “keep anything that lowers OFV.” The rule might use a chosen likelihood-based criterion or an information criterion, together with scientific and data-support requirements. The appropriate statistical tests and their calibration are a separate topic; the important point here is to specify the criterion before inspecting the search results and apply it consistently.

Forward Inclusion

In forward selection, first define a base model and a finite set of candidate covariate relationships. Each candidate is tested as a specific addition to the current model. If one addition meets the inclusion rule, retain it and compare the remaining candidates conditional on the updated model. Continue until no remaining candidate meets the rule, or until a prespecified stopping condition is reached.

Backward Deletion

Backward selection starts with a model that includes the candidate relationships being considered. For each relationship, fit the model with that relationship removed and compare it with the current fuller model. A small increase in OFV after deletion means that removing the relationship costs little likelihood fit; whether that is enough to remove it depends on the prespecified rule and scientific context. After a deletion, reassess the remaining terms in the updated model rather than relying only on their initial deletion results.

What Makes Selection Unstable?

Forward and backward searches are algorithms for exploring a set of models, not guarantees of a unique or robust final model. Their results can depend on the entry or deletion order, the candidate list, the criterion, and the data available. In particular, correlated covariates may carry overlapping information: one can appear useful when tested alone and contribute much less after another is included. The order in which a search evaluates candidates can then affect which relationship is retained.

A careful analysis should protect against common sources of instability:

• Define candidates before searching. Specify each covariate's form, target PD parameter, and allowable range of application. Avoid trying many undocumented transformations and retaining only the most favorable result.
• Keep model comparisons controlled. Use the same observations and the same non-covariate model components. Document changes to estimation settings, and do not compare OFVs from incompatible fits as if they reflected only a covariate change.
• Require estimable, converged fits. Check convergence, parameter plausibility, and whether estimates are unreasonably imprecise or on a boundary. An apparent improvement from a failed or poorly identified fit is not a sound reason to retain a term.
• Check order sensitivity. Compare reasonable forward and backward paths, or reconsider candidate ordering, when covariates are related. Mark terms whose retention depends strongly on the path as uncertain.
• Respect data support and scientific rationale. A mathematically favorable result does not compensate for sparse representation, an implausible relationship, or extrapolation beyond the covariate values observed.
• Report the search, not just its endpoint. Record the starting model, every tested addition or deletion, the fit criterion, decision threshold, and any failed fits. This makes the selection reproducible and exposes borderline decisions.

A practical alternative to automatic stepwise selection is to prespecify a small scientifically motivated model and assess whether its relationships are supported by the data. Forward search is economical when the base model is trusted and few additions are plausible. Backward search is useful when a defensible, estimable full candidate model can be fitted. A bidirectional search allows additions and deletions after each step, but increases the number of opportunities for data-driven selection. Information criteria can compare models while penalizing complexity, but they answer a criterion-specific model-ranking question; they do not eliminate uncertainty or establish biological truth. No single approach replaces clear rationale, model evaluation, and transparent reporting.

Common Mistakes and Limitations

Stepwise selection also tends to make the chosen model look more certain than it is: the same data are used both to select relationships and to estimate them. Coefficients and standard errors from the final selected model do not, by themselves, account for all the uncertainty introduced by the search. A selected relationship should therefore be reported as conditional on the candidate set, model assumptions, data, and selection rule. It is an estimated association in the model, not proof of a causal mechanism. Sensitivity analyses and independent scientific judgment remain important, especially when alternative paths lead to different models.