Backward Elimination With Prespecified Retention Thresholds

Why Backward Elimination Needs a Declared Rule

A full covariate model can contain effects that contribute little once the other effects are present. Backward elimination evaluates those effects conditionally: at each step, it asks whether removing one effect from the current model causes enough loss of fit to justify retaining it. The answer depends on the other effects still in that model, so the procedure is a sequence of comparisons, not a set of independent decisions made once at the start.

A prespecified retention threshold makes the search reproducible. It defines how much evidence against deletion is required to keep an effect, and it prevents a modeller from changing the rule after seeing which effects pass. This is a model-building decision rule; it does not by itself establish that a retained covariate has a causal effect or that the final model is uniquely best.

Likelihood-Ratio Deletion Tests

Let \(M\) be the current model and \(M_{-j}\) the model formed by removing effect \(j\). The reduced model is nested within the current model: it is obtained by constraining the parameters for effect \(j\) to their null values while leaving the other model components unchanged. For a likelihood-ratio test, fit both models to the same observations using the same likelihood definition and comparable estimation settings. In a population PK analysis, a common comparison uses the objective function value (OFV), defined as \(-2\) times the maximized log likelihood.

The deletion null hypothesis is that the parameters contributed by effect \(j\) are all zero (or at their specified null values), conditional on the other effects in \(M\). The alternative is that at least one of those parameters differs from its null value. Let \(q_j\) be the number of independent parameters constrained under the deletion null. Under the null and standard likelihood-ratio regularity conditions, the statistic has an approximate chi-square distribution with \(q_j\) degrees of freedom.

The statistic is nonnegative when both nested models have been successfully maximized on the same likelihood: the more flexible current model can reproduce the reduced model by setting the tested effect to its null value, and can potentially fit better. The p-value is \(P(\chi^2_{q_j} \geq \Lambda_j)\). For a prespecified p-value retention threshold \(\alpha_{\mathrm{ret}}\), retain an effect when its conditional deletion p-value is less than \(\alpha_{\mathrm{ret}}\); otherwise it is eligible for removal. Equivalently, retain when \(\Lambda_j\) exceeds the upper-tail chi-square critical value \(\chi^2_{q_j,\,1-\alpha_{\mathrm{ret}}}\). Equality at the threshold should be handled by a stated convention; a simple convention is to treat equality as eligible for deletion.

This chi-square calibration is an approximation, not an automatic property of every model comparison. It relies on regularity conditions, including an appropriately specified likelihood, identifiable parameters, and a null value that is not a problematic boundary. The tested effect should be represented by the stated number of regular parameters, and both fits should converge to comparable maxima. If these conditions are doubtful—for example, if a test concerns a parameter on a boundary—the usual chi-square reference distribution may not be justified, and the threshold-based decision needs an appropriate alternative calibration or a qualified interpretation.

How to Run and Document the Search

The threshold applies anew at every step. After an effect is removed, refit the resulting current model and test each remaining removable effect against that refitted model. A deletion statistic from the initial full model does not answer the later conditional question. In particular, an effect that passes its initial test may fail after a correlated or otherwise related effect is removed, and the reverse can also occur.

For a common search rule, the effect with the largest p-value is removed when that p-value is at least \(\alpha_{\mathrm{ret}}\). This selects the weakest conditional evidence for retention at that step. It is important to say which rule is being used: a procedure that removes any eligible effect according to a fixed ordering can follow a different path. Whichever rule is prespecified, the sequence of current models and the result of every deletion test should be recorded.

Thresholds, Degrees of Freedom, and Ties

The p-value threshold and the likelihood-ratio cutoff are two ways of expressing the same decision rule, provided the degrees of freedom are correct. For one degree of freedom, a 0.05 upper-tail threshold corresponds to a chi-square critical value of approximately 3.841. For more than one parameter, the critical value changes. Counting an effect as one “covariate” does not necessarily mean its deletion has one degree of freedom; the degrees of freedom are the number of independent parameters constrained by that deletion. The earlier material on degrees of freedom for covariate tests gives the general parameter-counting framework.

When multiple eligible effects have exactly the same p-value, a search rule based only on the largest p-value does not identify a unique deletion. Before fitting the search, specify how to break such ties. One defensible rule is to remove the tied effect with lower prespecified scientific priority, keeping the more important effect for the next conditional comparison. For near ties, define a numerical tolerance in advance and apply the same tie rule whenever the difference is within that tolerance. Report the tied results and the rule used; do not let software ordering decide silently.

Diagnostics and Common Failure Modes

Check the numerical and statistical basis of each deletion before acting on its p-value. A reduced-model OFV lower than the current-model OFV gives a negative likelihood-ratio statistic, which contradicts the expected ordering for successfully maximized nested likelihoods. Small apparent violations can arise from numerical optimization error, different likelihood settings, or fits that have not reached comparable maxima. Do not truncate a negative statistic to zero and proceed as if the comparison were sound; investigate the fits, improve or repeat estimation, and document any unresolved limitation.

• Incomparable fits: different observations, likelihood definitions, or estimation settings can make an OFV difference uninterpretable as the intended nested-model test.
• Incorrect degrees of freedom: using one degree of freedom for an effect that removes multiple independent parameters gives the wrong reference distribution and can change the decision.
• Stale conditional tests: using p-values from the initial model after a deletion ignores the changed conditioning set. Refit and retest the remaining effects.
• Unstable fits: poor convergence, implausible estimates, or sensitivity of OFV to starting values weakens confidence in the comparison. Investigate numerical stability before treating a small difference as meaningful.
• Unstated tie handling: when results tie or nearly tie, an undocumented software ordering can make the selected path difficult to reproduce. Apply the declared tie rule and report it.

A frequent mistake is to regard \(p\geq\alpha_{\mathrm{ret}}\) as proof that an effect is absent, or to treat \(p<\alpha_{\mathrm{ret}}\) as proof that it is scientifically important. The p-value describes the data’s compatibility with the deletion null under the test assumptions; the threshold is a prespecified search rule. It is not a measure of clinical importance, and the final selected model reflects the candidate set, threshold, and sequence of conditional comparisons.

How Backward Elimination Differs From Other Choices

Forward addition, covered in the earlier material on forward addition with conditional covariate effects, begins from a smaller model and tests additions. Backward elimination begins from a larger specified model and tests deletions. Because each evaluates effects conditional on a different current model at each step, the two procedures need not reach the same model. A single-effect Wald test can be less computationally demanding than refitting a reduced model, but it is not the same likelihood-ratio comparison and may behave differently in a nonlinear mixed-effects model. The objective-function methods considered earlier also help distinguish a genuine fit difference from numerical noise.

Backward elimination has practical limitations. It requires fitting the initial full model and multiple reduced models, and a poorly estimated or over-parameterized full model can make the entire path unreliable. The threshold does not account for the full search’s repeated decisions, and the selected model should not be interpreted as if it had been chosen by one isolated test. Those concerns motivate separate treatment of search-wide testing and alternative model-selection criteria; they do not change the arithmetic or conditional logic of the procedure defined here.