Shrinkage After Covariate Conditioning

Why Conditioning on Covariates Matters

A population model does not usually assign every individual the same typical parameter. It may predict a different typical clearance for individuals with different body weights, for example, while allowing additional individual deviations through random effects. Shrinkage concerns how strongly those random-effect deviations are pulled toward their population distribution when an individual’s data provide limited information. The relevant deviation is therefore measured after accounting for the covariates, not against one common value for everyone.

This distinction also matters when comparing groups. If groups differ in covariate composition, their typical parameters may differ even when their random effects have the same conditional distribution. Treating such typical differences as unexplained random-effect variation can change apparent shrinkage without any change in the information supplied by each individual’s observations.

Typical Parameters and Conditional Random Effects

Let \(x_i\) denote the covariates for individual \(i\), let \(\beta\) denote fixed-effect parameters, and let \(\eta_i\) denote that individual’s random effect or vector of random effects. A general NLME model represents the individual parameter as \(\theta_i=\theta_{\mathrm{typ}}(x_i;\beta,\eta_i)\). For a positive parameter modeled on the log scale, a common form is:

Here \(\theta_{\mathrm{typ}}(x_i;\beta)\) is the parameter value at \(\eta_i=0\), for the covariate profile \(x_i\). Under the conventional centered random-effects model, \(\eta_i\mid x_i\sim N(0,\Omega)\), with \(\Omega\) the random-effects covariance matrix. This assumption means the random effects have conditional mean zero; it is stronger than merely asserting that their mean is zero in the overall population.

For a scalar normally distributed \(\eta_i\), its conditional median is zero. Consequently, the median of \(\theta_i\) given \(x_i\) is \(\theta_{\mathrm{typ}}(x_i;\beta)\) in the exponential model. Its arithmetic mean is different:

The median statement relies on the conditional median of \(\eta_i\) being zero, as it is under the centered normal assumption. A zero conditional mean alone does not imply a zero conditional median. Thus, without the additional distributional assumption, neither the conditional median nor the conditional arithmetic mean of \(\theta_i\) can generally be identified with \(\theta_{\mathrm{typ}}(x_i;\beta)\).

Why the Residual Eta Is the Relevant Quantity

After conditioning on covariates, the random effect represents the individual’s deviation from their own covariate-based typical parameter. In a simple log-parameter representation, define \(z_i=\log(\theta_i)\) and \(\mu(x_i)=\log\{\theta_{\mathrm{typ}}(x_i;\beta)\}\). Then \(z_i=\mu(x_i)+\eta_i\). The quantity relevant to the random effect is the residual \(z_i-\mu(x_i)\), not \(z_i\) minus a single population-wide reference when \(\mu(x_i)\) varies with \(x_i\).

The following derivation gives a transparent version of the shrinkage mechanism. It uses a Gaussian random-intercept model for a log-parameter summary. Assume that, conditional on known covariates and fixed effects, \(r_i=z_i-\mu(x_i)=\eta_i+e_i\), where \(\eta_i\sim N(0,\omega^2)\), \(e_i\sim N(0,s_i^2)\), and \(\eta_i\) and \(e_i\) are independent. The term \(e_i\) represents measurement uncertainty in the summary. These assumptions give a normal conditional distribution for \(\eta_i\) given \(r_i\).

Collecting the terms involving \(\eta_i\) in the exponent gives a quadratic with precision \(1/\omega^2+1/s_i^2\). Its conditional variance is therefore \((1/\omega^2+1/s_i^2)^{-1}\). The center of that quadratic, and hence the conditional mean and mode, is:

The multiplier \(\lambda_i=\omega^2/(\omega^2+s_i^2)\) lies between zero and one. It weights the covariate-adjusted residual toward the data: when the individual summary is precise, \(s_i^2\) is small and \(\lambda_i\) is near one; when it is imprecise, the estimate is pulled toward the conditional prior mean zero. This is a Gaussian illustration, not an exact general formula for nonlinear models, where the likelihood and posterior may not be Gaussian.

Covariate Imbalance Can Mimic Random-Effect Heterogeneity

Suppose a binary covariate affects the typical log parameter, but the fitted model omits it. Individuals in the two covariate groups then have different expected log parameters. An omitted covariate effect can be absorbed into what appears to be a random effect. The resulting apparent random-effect distribution can have group-specific means and extra variance, even if the true random effects have mean zero and the same variance within both groups.

This is not a claim that every covariate imbalance changes an estimated shrinkage value in the same direction. The result depends on the covariate effect, its distribution, the data precision, and the estimation procedure. The simple calculation below isolates the mechanism under known variances and a Gaussian summary model.

Assumptions, Diagnostics, and Alternatives

The interpretation of conditional shrinkage relies on a coherent covariate model and a specified conditional random-effects distribution. In the conventional formulation, random effects are centered conditional on the modeled covariates, often with \(\eta_i\mid x_i\sim N(0,\Omega)\). If the conditional mean of \(\eta_i\) varies with a covariate, the model’s fixed-effect and random-effect components no longer have the stated interpretation. Covariates must also be measured and coded appropriately, and the functional form used for their effects must be adequate over the observed range.

Useful checks focus on whether covariate conditioning is credible and whether comparisons use like-for-like quantities:

• Compare covariate distributions across the groups or subsets whose shrinkage summaries are being contrasted. Large imbalances make an unadjusted comparison difficult to interpret.
• Check that the covariate function is plausible over the data-supported range. A misspecified trend or omitted interaction can leave systematic structure in what the model treats as random effects.
• Examine whether the fitted conditional random-effects distribution is consistent with the assumed centering and variance structure across important covariate ranges. Persistent conditional shifts suggest that the conditional model may be inadequate.
• When reporting shrinkage summaries, state which conditional random-effect variance and which set of individuals were used. A pooled summary can change simply because the covariate mix changes.

The main alternatives are not interchangeable. Ignoring a covariate leaves its systematic effect available to be absorbed into random effects. Modeling the covariate as a fixed effect defines a typical parameter that changes with \(x_i\), leaving \(\eta_i\) to represent residual individual variation under the model. Stratifying analyses can avoid some pooling across very different covariate profiles, but may discard information, produce imprecise estimates in small strata, and fail to model gradual trends. A covariate-adjusted model is useful only when its functional form and data support are adequate; it does not automatically remove misspecification.

Shrinkage summaries based on estimated random effects are also finite-sample diagnostics, not exact measurements of individual information. Estimated fixed effects, estimated variance components, nonlinear likelihoods, and a limited number of individuals can all affect the result. Conditioning clarifies the target of the calculation, but it does not eliminate uncertainty in the fitted model or make weakly supported covariate effects reliable.

Common Mistake

Wrong reasoning: “The raw parameter values differ between two groups, so the group with larger values must have larger random effects or less shrinkage.” This compares observed parameters to a common reference even when the model predicts different typical values for the groups.

Correction: First evaluate each individual’s typical parameter at their covariate values. Assess random-effect deviations relative to those conditional predictions. A difference in typical values caused by covariate composition is not, by itself, evidence of a difference in conditional eta distributions or individual data information.