Choosing Between Correlated Covariates

Why Correlated Covariates Require Deliberate Choice

Body weight and age, or renal function and a related laboratory measure, may carry overlapping information about a pharmacokinetic parameter. If two covariates move together in the study population, a model may predict well while being uncertain about how much of the association belongs to each covariate separately. The challenge is therefore not simply to identify which candidate gives the smallest p-value or enters a search first. It is to decide which covariate representation is scientifically defensible, estimable from the available data, and useful for the intended predictions.

The question is conditional: does a covariate contribute information about the parameter after accounting for the other covariates in the model? A strong relationship between a covariate and an outcome when considered alone need not imply a precise, distinct effect once a correlated covariate is included. Conversely, a weak conditional estimate can reflect limited information for separating two predictors rather than evidence that neither is scientifically relevant.

Model and Assumptions for a Fair Choice

For illustration, suppose a subject-specific clearance parameter is modeled on the log scale. For subject \(i\), let \(x_{Wi}\) be centered body weight in units of 10 kg and \(x_{Ai}\) be centered age in units of 10 years. A candidate model containing both is

Here \(CL_{\mathrm{ref}}\) is typical clearance at the chosen covariate reference values, \(\beta_W\) and \(\beta_A\) are fixed covariate coefficients, \(\eta_i\) is a subject-level random effect, and \(Y_{ij}\) is the observation at time \(t_{ij}\). The function \(f\) represents the pharmacokinetic model. In this specification, a one-unit increase in \(x_{Wi}\), holding age fixed, multiplies typical clearance by \(\exp(\beta_W)\). The same interpretation applies to age through \(\exp(\beta_A)\). These are conditional model interpretations, not claims that changing a person's weight or age would cause the predicted change.

A meaningful comparison presumes that the candidate models use the same subjects and observations, the same structural pharmacokinetic model, and comparable random-effects and residual-error assumptions. Estimation should be successful and the likelihoods should refer to the same data and likelihood convention. If these conditions differ, an apparent improvement may be due to the changed analysis rather than the covariate choice. Covariate measurements should also be sufficiently reliable and their units and reference values clearly defined.

Correlation itself does not violate the model. It does, however, make conditional effects harder to estimate. The following short derivation explains the basic mechanism in a simple linear setting; in a nonlinear mixed-effects model (NLME), the same issue appears through the information available in the likelihood, though the exact precision depends on the full model and design.

Its inverse is

Thus the coefficient variances are inflated by \(1/(1-r^2)\), and their standard errors by \(1/\sqrt{1-r^2}\). As \(|r|\) approaches 1, the information matrix becomes nearly singular: many combinations of the two coefficients can produce similar fitted values. This is a useful explanation, not an exact NLME standard-error formula. In an NLME model, local likelihood curvature, within-subject sampling, random effects, and the structural model also determine precision.

Combining Scientific Relevance, Precision, and Fit

Scientific relevance helps determine which conditional effect is meaningful to estimate. Consider whether a candidate covariate has a plausible relationship to the parameter, whether it is measured at the relevant time, and whether its intended use matches the model interpretation. A covariate that is merely a proxy for another may be useful for prediction in a similar population, yet less transportable if the relationship between the proxy and the underlying physiology changes.

Data support matters just as much. Examine the covariates' joint distribution, their range, and whether there are enough subjects with differing combinations of their values. If nearly every heavier subject is also older, the data provide few comparisons of heavier versus lighter subjects at the same age. The joint coefficients may consequently have large standard errors, unstable signs, or strong dependence on modeling choices. A small standard error in a single-covariate model does not resolve this issue: that model's coefficient can absorb the association carried by both predictors.

Fit statistics can summarize how well candidate models balance fit and complexity, but they are not substitutes for these judgments. For example, the Akaike information criterion is \(\mathrm{AIC}=2k-2\ell\), where \(k\) is the number of estimated parameters and \(\ell\) is the maximized log likelihood. When models are fit comparably to the same data, a smaller AIC is evidence favoring the model's expected information performance under the criterion; it is not a probability that the model is true. Small differences should not be treated as decisive, particularly when the scientific interpretations differ.

Prediction as a Separate Criterion

A covariate can be useful for prediction even when its conditional coefficient is difficult to distinguish from that of a correlated alternative. Conversely, a model with a slightly better fit to the estimation data may not predict better for new subjects. Prediction should be assessed for the target of interest and on observations not used to estimate or choose the model. For example, predicting typical clearance from covariates for new subjects is different from predicting their individual concentration profiles after their own concentration data have been observed.

One simple measure for predictions of log clearance is root mean squared error (RMSE): for \(m\) evaluation subjects, it is the square root of the average squared difference between observed and predicted log clearance. The compared predictions must target the same quantity and be evaluated on the same subjects. An evaluation set that is too small or unlike the intended population can make apparent differences uninformative. Prediction error also does not by itself establish that a covariate's coefficient has a stable mechanistic interpretation.

Practical Comparison and Failure Modes

The main alternatives are to use one covariate, include both, or use a scientifically justified representation that combines or replaces correlated measurements. A one-covariate model is often easier to interpret and may be more precise, but its coefficient can represent the combined association of omitted correlated predictors. A joint model can estimate distinct conditional effects when the data contain enough independent variation, but may be imprecise when they do not. A composite measure or a mechanistically motivated alternative can be appropriate when it has a defensible scientific meaning and is defined before interpreting its fit; combining variables solely to make a preferred result appear is not a sound solution.

Common warning signs include large standard errors in the joint model, estimates that change markedly between single and joint models, implausible conditional interpretations, and nearly indistinguishable fit or prediction results across alternatives. These signs call for cautious conclusions. They are not proof of a software failure. Check convergence, parameter uncertainty, the covariate distributions, and whether the fit is genuinely comparable. If the data cannot separate the covariates, report that limitation and avoid presenting a weakly identified conditional effect as a precise scientific finding.

A frequent mistake is: “Weight entered the stepwise model first, so weight is the important covariate and age can be ignored.” Entry order is not evidence that one predictor is uniquely responsible for the association, especially when the candidates are correlated. The correction is to compare scientifically defensible models using the same data and modeling assumptions, inspect conditional precision, and consider performance for the intended prediction target. Selection results are evidence to weigh, not a verdict about biological importance or causation.