Separating Covariate Effects From Random Effects

Why Separate Covariates From Random Effects?

People with different covariate values can have systematically different typical PD parameters. Even among people with the same covariate values, however, the parameter may vary from one person to another. A population model represents these as distinct components: a covariate effect shifts the conditional population value, while a random effect represents remaining between-subject variation around that value.

This distinction matters for interpretation. A covariate coefficient describes how the model’s typical parameter changes with the covariate; it does not describe every individual’s change. A random-effect variance describes the model’s residual heterogeneity on a specified parameter scale; it is not itself a covariate effect. Both components can be present, and neither should be inferred solely from the other.

Model Components and Their Assumptions

Begin with an additive model for a positive or unrestricted parameter \(P_i\), with a centered covariate \(z_i\):

Here, \(\theta\) is the population value at \(z_i=0\), \(\beta\) is the change in the conditional population mean for a one-unit increase in \(z_i\), and \(\omega^2\) is the conditional random-effect variance. The covariate’s units determine the units of \(\beta\); \(\theta\), \(\beta z_i\), and \(\eta_i\) must all have the units of \(P_i\). Under the stated conditional mean and variance assumptions, \(E(P_i\mid z_i)=\theta+\beta z_i\) and \(\operatorname{Var}(P_i\mid z_i)=\omega^2\).

The random effect is often assumed to follow a normal distribution, \(\eta_i\sim N(0,\omega^2)\), though the normality assumption is stronger than the mean and variance conditions above. In either case, the zero conditional mean is important: it makes the covariate term the systematic shift and the random effect a deviation around that shift. The model also assumes the specified covariate function is appropriate and that the random-effect distribution adequately represents residual subject differences. If random effects are systematically associated with a covariate in a way the model does not represent, the covariate and random-effect components may not be cleanly separated.

For a positive parameter, an exponential model is often more appropriate:

Here \(\theta\) is the typical parameter at \(z_i=0\), and \(\exp(\beta)\) is the multiplicative change in the typical parameter for a one-unit increase in \(z_i\). The conditional median of \(P_i\) is \(\theta\exp(\beta z_i)\); because the random effect is normal on the log scale, the conditional arithmetic mean is \(\theta\exp(\beta z_i+\omega^2/2)\). Thus, “population mean” must be interpreted on the scale specified by the model: the covariate shifts the mean on the log scale, while the arithmetic mean on the original scale also depends on the random-effect variance.

Why Covariate Variation and Random-Effect Variation Differ

The distinction can be made precise using the law of total variance. For the additive model, suppose the covariate \(Z\) varies across subjects, \(E(\eta\mid Z)=0\), and \(\operatorname{Var}(\eta\mid Z)=\omega^2\). Then the variance of \(P\) across the whole population includes both variation in the covariate-predicted mean and variation around that mean.

The first term, \(\omega^2\), is the average within-covariate conditional variance. The second term is variation in the conditional mean as \(Z\) varies. This decomposition follows because, at a given \(Z\), the additive model has conditional variance \(\omega^2\), while its conditional mean is \(\theta+\beta Z\). It does not say that all population variation is random-effect variation, nor that adding a covariate must reduce the conditional variance by a particular amount.

In practice, random-effect variance estimates can change when a covariate is added. If the covariate captures a systematic pattern that was previously left in the unexplained subject variation, the estimated residual variance may decrease. But the estimate also depends on the data, model structure, and parameter scale. A decrease is evidence about the fitted model’s residual heterogeneity, not a direct or exact measure of the fraction of biological variation “explained.”

Interpreting Random Effects on a Log Scale

For the exponential model, the covariate shifts the typical parameter multiplicatively, while \(\eta_i\) acts multiplicatively on the original scale through \(\exp(\eta_i)\). A fixed \(\omega^2\) means constant random-effect variance on the log scale, not constant variance in the original parameter units. In fact, under the lognormal model:

Consequently, original-scale variance can be larger at covariate values with larger typical parameters, even when \(\omega^2\) is unchanged. A comparison of random-effect variances is meaningful only with attention to the scale and parameterization used.

Reading Changes in Estimated Variance

When comparing a model without a covariate to one with it, first check that both models use the same parameter scale and the same random-effect structure. Then interpret the covariate coefficient on its stated scale and compare the estimated random-effect variances. For a variance estimate \(\omega^2\), the corresponding random-effect standard deviation is \(\omega=\sqrt{\omega^2}\), which is often easier to discuss in the parameter’s units (or log units).

For example, the variance might fall substantially while remaining positive. That indicates less estimated residual heterogeneity under the covariate model, not that subjects have become less variable in reality. Parameter uncertainty also matters: estimates can vary with sampling, and a numerical difference alone does not establish that the covariate is reliably associated with the parameter.

Diagnostics, Alternatives, and Limitations

A sensible interpretation depends on whether the model can distinguish the systematic covariate pattern from subject-specific deviations. Examine whether the fitted conditional mean changes in the expected direction across covariate values and whether residual subject-level deviations remain centered around that mean. Review estimation uncertainty and whether the random-effect variance is stable under reasonable model fits. A variance estimate near zero, very large uncertainty, or unstable estimates can indicate that the data do not support precise separation of the components.

Common warning signs include a covariate coefficient that changes markedly across model specifications, a random-effect variance that changes dramatically without a clear model explanation, or systematic residual patterns across the covariate range. Such patterns can arise when the assumed covariate relationship is inadequate, when data are sparse, or when the chosen parameter scale is inappropriate. These signals call for investigation; they do not identify one cause automatically.

The main alternatives are not interchangeable interpretations. A model with random effects but no covariate allows unexplained subject heterogeneity but does not represent a systematic mean shift with the measured covariate. A covariate-only model represents a conditional mean pattern but assumes no remaining random heterogeneity in that parameter. A model with both components allows a systematic shift and residual variation, but requires enough information to estimate both. The appropriate choice depends on the scientific model and available data, not merely on which fit produces a smaller variance estimate.

A common mistake is to say, “The random-effect variance fell by 49%, so the covariate explains 49% of between-subject variability.” This overstates what the comparison establishes. The calculation is a relative change in an estimated variance under two fitted models. It is not a direct causal or biological allocation of variation, and it does not account by itself for uncertainty or changes in model assumptions. Report the coefficient, its units and scale, the variance estimates on the same scale, and the limits of the comparison.