Why the Search Path Matters
Covariate selection is not simply a ranking of candidates: the evidence for one candidate can change when another candidate is included in the model. This is especially relevant when candidates are correlated, because they may explain overlapping patterns in the data. A search procedure therefore evaluates effects conditional on the model reached so far, and its route through possible models can influence the result.
Forward addition starts from a base model and adds candidates according to conditional tests. Backward elimination starts from a fuller model and removes candidates according to tests conditional on the others. Bidirectional search permits both additions and removals as the model changes. Each method can be useful, but they need not select the same model: they examine different sequences of nested comparisons and may use different entry and retention thresholds.
Likelihood Comparisons Along a Path
Let the objective function value (OFV) be \(-2\) times the maximized log likelihood, so a smaller OFV indicates a better fit. If a reduced model is nested within a full model, the likelihood-ratio statistic is the reduced-model OFV minus the full-model OFV. Under the usual regularity conditions, its null distribution is approximately chi-square, with degrees of freedom equal to the difference in the number of freely estimated parameters.
The comparison requires the models to be fitted to the same observations, to be nested, and to differ in the specified covariate effect rather than in unrelated structural or random-effects assumptions. The likelihood fits must also be sufficiently reliable for the OFV difference to be meaningful. For a one-parameter effect, a common illustrative reference is the 5% chi-square cutoff \(3.841\). A search may instead prescribe a more stringent entry cutoff and a separate retention cutoff; the numerical rules must be stated before interpreting its decisions.
The reason conditioning changes the comparison follows directly from the OFV differences. For two candidates \(A\) and \(B\), the evidence for \(B\) without \(A\) is \(\mathrm{OFV}(M_0)-\mathrm{OFV}(M_B)\), whereas its evidence given \(A\) is \(\mathrm{OFV}(M_A)-\mathrm{OFV}(M_{AB})\). These are different nested comparisons, so they need not yield the same statistic.
Here \(M_0\) is the base model, \(M_A\) and \(M_B\) add one candidate each, and \(M_{AB}\) includes both. Correlation can make the candidates share explanatory information, so the conditional evidence for either may be smaller than its evidence alone. Other data patterns can make conditional evidence larger. The change is not a change in the data; it is a change in the model comparison being asked.
Worked Example: Calculating Conditional Evidence
Suppose a hypothetical base model has OFV 1000. Adding candidate \(A\) gives OFV 992, adding candidate \(B\) alone gives 993, and including both gives 988. Assume each candidate adds one parameter, the fits are reliable, and the models are nested. Calculate the evidence for each candidate alone and conditionally, using \(3.841\) as the illustrative one-degree-of-freedom cutoff.
Alone: for \(A\), \(\mathrm{LRT}=1000-992=8\). For \(B\), \(\mathrm{LRT}=1000-993=7\). Both exceed \(3.841\).
Conditional on \(A\): the statistic for adding \(B\) is \(992-988=4\), which exceeds \(3.841\).
Conditional on \(B\): the statistic for adding \(A\) is \(993-988=5\), which also exceeds \(3.841\). Thus, in this hypothetical example, both candidates have evidence beyond the illustrative cutoff in each stated comparison. The calculations do not establish that either covariate has a causal effect; they quantify improvement in these fitted nested models.
How the Three Search Paths Differ
The procedures differ in their starting point and in which conditional comparisons they make first. A forward procedure usually adds the eligible candidate with the strongest evidence, then reevaluates remaining candidates against the enlarged model. A backward procedure starts with a specified full candidate set and removes the least-supported effect if it fails the retention rule; the remaining effects are then retested in the smaller model. A bidirectional stepwise procedure allows both directions, for example by adding a candidate and then checking whether an included candidate can now be removed.
“Bidirectional” does not specify a unique algorithm. Some implementations reconsider all excluded candidates after every change; others use a fixed sequence or stop after a prescribed number of passes. Entry and removal cutoffs may differ. Consequently, a reproducible description must give the starting model, eligible effects, test and degrees of freedom, thresholds, order or tie rule, and stopping rule.
Worked Example: Different Rules Produce Different Final Models
Consider hypothetical OFVs of 1000 for \(M_0\), 992 for \(M_A\), 993 for \(M_B\), and 988 for \(M_{AB}\). Suppose forward entry requires an LRT of at least 6, while backward retention requires an LRT of at least 3.841. Both effects add one parameter; all comparisons use the same data and reliable nested fits.
Forward path. From \(M_0\), \(A\) has LRT \(1000-992=8\), and \(B\) has LRT \(1000-993=7\). The procedure adds \(A\), the stronger candidate. Given \(A\), the LRT for adding \(B\) is \(992-988=4\), below the entry cutoff of 6. Forward search stops at \(M_A\).
Backward path. Start from \(M_{AB}\). Removing \(A\) gives the comparison \(993-988=5\); removing \(B\) gives \(992-988=4\). Both exceed the retention cutoff of 3.841, so neither is removed. Backward search retains \(M_{AB}\).
The final models differ even though the OFVs and arithmetic are the same. Forward search applies a stricter entry rule to \(B\) after \(A\) is present; backward search asks whether either effect meets the stated retention rule in the full model. This is a consequence of the procedures and thresholds, not evidence that one calculation is inconsistent.
A bidirectional procedure could, for example, add \(A\) under the entry rule, then test whether \(A\) remains supported after another candidate is added. If it revisits all excluded candidates and uses the same rules, it may reach a different endpoint from either one-way procedure. It can also revisit a model already seen. Practical implementations should specify how they avoid cycling—for example, by recording visited models and stopping when a full pass makes no change.
Worked Example: A Strong Marginal Candidate Weakens After Conditioning
In another hypothetical search, let the OFVs be 600 for \(M_0\), 594 for \(M_A\), 592 for \(M_B\), and 590 for \(M_{AB}\). Both candidates add one parameter. Use an entry cutoff of 3.841 and assume the same observations, nested models, and reliable fits throughout.
From the base model, \(A\)'s LRT is \(600-594=6\), while \(B\)'s is \(600-592=8\). A forward procedure therefore adds \(B\) first. Once \(B\) is included, the conditional statistic for \(A\) is \(592-590=2\), below the cutoff, so \(A\) is not added.
The corresponding backward comparison also shows why \(A\) is weak conditional on \(B\): removing \(A\) from \(M_{AB}\) increases OFV by \(592-590=2\). In contrast, the evidence for \(B\) given \(A\) is \(594-590=4\), above the cutoff. In this example, \(A\) appears supported when tested alone but contributes less once \(B\) is included. The conclusion is about these conditional model comparisons, not a general claim that \(A\) is irrelevant.
Assumptions, Diagnostics, and Failure Modes
A search path is interpretable only if its component comparisons are meaningful. The relevant assumptions and checks include:
- Comparable fits: compare models fitted to the same records and likelihood definition. A change in data handling or likelihood approximation can make an OFV difference reflect more than the candidate effect.
- Valid nesting and degrees of freedom: the reduced model must be obtained by imposing the restrictions represented by the tested covariate effect. Count the added parameters correctly; a multi-parameter effect does not use a one-degree-of-freedom reference.
- Reliable optimization: check convergence, boundary estimates, parameter plausibility, and sensitivity to starting values. Numerical noise can reverse close decisions, especially near a cutoff.
- Stable path rules: record candidate order, tie handling, thresholds, and stopping rule. Repeating the search with a different order or start can reveal dependence on those choices.
- Substantive plausibility: inspect whether retained effects are interpretable and whether the fitted model remains adequate. A search decision alone is not a diagnostic of the full pharmacokinetic model.
A common failure is to report only the final model and treat it as if every included covariate had passed the same test under the same conditioning set. That reasoning is wrong: a candidate may have entered when few effects were present and later become weak, or may be retained in a backward search because it contributes conditionally in a fuller model. The correction is to report the path and the exact comparison supporting each decision, then interpret final effects conditional on the final model.
Another failure is to equate different endpoints with a contradiction or to choose whichever endpoint is preferred after seeing the results. Different starting models, thresholds, or orders define different procedures. Pre-specify the rules, report the sensitivity of conclusions to reasonable alternatives, and avoid implying that a selected model is uniquely established by the search.
Comparing the Main Approaches
| Approach | Starting point | Primary strength | Important limitation |
|---|---|---|---|
| Forward addition | Base model | Does not require fitting the largest candidate model first | A candidate that is weak early may not be considered again unless the algorithm explicitly revisits it |
| Backward elimination | Full candidate model | Assesses effects conditional on the other included candidates at the start | Requires a suitable full model to be estimable and depends on its specified candidate set |
| Bidirectional search | Often the base model, with additions and removals | Can reconsider included and excluded effects as the model changes | Rules, revisiting, and stopping must be defined to prevent ambiguous paths or cycling |
None of these approaches removes the need for scientific judgment or makes the selected model independent of the candidate set. Path comparison is useful as a stability assessment: if modest changes in order or starting point produce different covariates, the data provide limited separation among the corresponding conditional explanations. That is a reason to report uncertainty and model dependence, not to present one path as proof of a unique biological mechanism.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A reduced model has OFV 420 and a nested full model has OFV 416. What is the LRT statistic? If one parameter was added, does it exceed the illustrative cutoff 3.841?
- Why can a candidate have a larger LRT when tested alone than when tested conditional on another covariate?
- In a search with entry cutoff 6 and retention cutoff 3.841, can a candidate fail entry but still be retained by a backward procedure? Explain.
- Name two details that must be reported to make a bidirectional search reproducible.
Solutions. (1) \(\mathrm{LRT}=420-416=4\), which is greater than 3.841. (2) The two tests compare different nested model pairs; correlated candidates can account for overlapping information, reducing the added candidate’s conditional contribution. (3) Yes. The rules differ, and backward search evaluates the candidate conditional on the other effects in its starting model; its conditional statistic may exceed 3.841 even if the candidate did not satisfy a forward entry rule of 6. (4) Examples include the starting model, candidate set, entry and removal thresholds, candidate order or tie rule, test degrees of freedom, and stopping rule.