Reading Shrunk Subject Trajectories

Why Subject Trajectories Are Shrunk

A longitudinal model separates a population-level biomarker pattern from subject-to-subject departures around that pattern. A subject with observations that appear unusually high or unusually steep may nevertheless receive a predicted subject effect closer to zero than the corresponding raw estimate. This is shrinkage: the prediction combines the subject’s data with the model’s estimate of how much subjects typically vary.

Shrinkage is not a correction that should be applied to every observed value, nor does it mean that subjects truly differ less than the data suggest. It is a feature of predicting a subject effect when the observations contain noise. The amount depends on how informative that subject’s observations are relative to the assumed between-subject and residual variation.

The Model and the Source of Shrinkage

For subject \(i\), write the longitudinal model as

Here \(\mathbf{y}_i\) is the vector of biomarker measurements, \(\mathbf{X}_i\boldsymbol{\beta}\) is the population-level mean at the subject’s observation times, and \(\mathbf{Z}_i\mathbf{b}_i\) represents the subject’s random deviation. The random effects and residual errors are independent, and residual errors are independent across subjects. The covariance matrix \(\mathbf{D}\) describes between-subject variation; \(\mathbf{R}_i\) describes within-subject residual variation. For the standard BLUP formula below, these covariance parameters are treated as known. In practice, fitted variance components are substituted.

Let \(\mathbf{r}_i=\mathbf{y}_i-\mathbf{X}_i\boldsymbol{\beta}\) be the subject’s observed outcomes after subtracting the population mean at each observation time. Under this model, the BLUP of the subject effect is

This formula makes the balance visible. The subject’s residual pattern \(\mathbf{r}_i\) provides information about their deviation, while \(\mathbf{D}\) and \(\mathbf{R}_i\) determine how much weight that information receives. If the observations are noisy or sparse, the subject-specific pattern is uncertain and the prediction leans more heavily on the assumed random-effects distribution, which is centered at zero.

A Scalar Derivation: Shrinkage of a Subject Mean

Consider the random-intercept model \(y_{ij}=\mu+b_i+\varepsilon_{ij}\), with \(b_i\sim N(0,\tau^2)\), independent errors \(\varepsilon_{ij}\sim N(0,\sigma^2)\), and \(n_i\) measurements for subject \(i\). Assume \(\mu,\tau^2,\sigma^2\) are known for this derivation. The subject’s raw mean is \(\bar{y}_i\), and its deviation from the population mean is \(\bar{y}_i-\mu\). The likelihood variance of this deviation is \(\sigma^2/n_i\), while the prior variance of \(b_i\) is \(\tau^2\).

The conditional density of \(b_i\) given the observations is proportional to the product of the normal prior density and the normal density of the subject mean. Keeping only terms involving \(b_i\), its log density is

Collecting the quadratic and linear terms in \(b_i\) gives conditional variance \(V_i=(1/\tau^2+n_i/\sigma^2)^{-1}\) and conditional mean \(V_i(n_i/\sigma^2)(\bar{y}_i-\mu)\). Therefore,

The multiplier is between zero and one when both variances are positive. The raw mean’s deviation is shrunk toward zero; equivalently, the predicted subject level \(\mu+\widehat{b}_i\) is pulled toward the population level \(\mu\). More observations reduce \(\sigma^2/n_i\), so the multiplier moves closer to one. Larger residual noise reduces it, while larger between-subject variance increases it.

Why Raw Means Can Miss a Trajectory

A raw subject mean collapses measurements across time into one number. It can describe an average level over the observed visits, but it does not say whether the biomarker rose, fell, or remained stable. It can also be affected by which times were observed: a mean based mostly on early visits may not be comparable with one based mostly on late visits when the population trajectory changes over time.

With random intercepts and slopes, a common model is

The population trajectory is \(\beta_0+\beta_1t\). The subject’s predicted trajectory adds \(\widehat{b}_{0i}+\widehat{b}_{1i}t\). A raw slope calculated from one subject’s observations need not equal the predicted random slope: the latter is informed by both that subject’s data and the fitted distribution of random effects. The following example isolates the slope calculation by centering time and assuming independent random intercept and slope effects.

Reading the Prediction and Checking Its Reliability

A predicted subject trajectory should be read as a model-based estimate of that subject’s deviation from the population trajectory, not as a smoothed copy of the measurements or a directly observed biological path. The population trajectory and the subject effects must be interpreted together. For example, a positive predicted random slope means a steeper trajectory than the population slope, not necessarily an increasing biomarker: if the population slope is sufficiently negative, the subject’s total slope can remain negative.

The amount of shrinkage is a model consequence, not a direct diagnostic of whether the model fits. Before interpreting individual trajectories, check whether the assumptions that drive the prediction are plausible. Useful questions include:

• Do residuals show patterns over time that suggest the assumed residual covariance is inadequate?
• Do observed subject patterns appear to vary in ways the chosen random-effects structure cannot represent, such as substantial curvature when only random slopes are included?
• Are some subjects observed at very few times, or only within a narrow time range? Their individual slopes may be weakly informed and strongly shrunk.
• Do a small number of extreme observations dominate a subject’s apparent level or change? A Gaussian model may be sensitive to such observations.
• Are fitted variance components plausible and estimable? If between-subject slope variation is estimated near zero, slope predictions will be close to zero for all subjects.

These checks do not establish that a particular prediction is correct. They can reveal that the assumptions used to calculate it are questionable. In particular, if observation times or follow-up depend on a subject’s unobserved biomarker process, the usual mixed-model account may not adequately represent the observation process. Informative visit schedules require attention to that process rather than an assumption that shrinkage resolves it.

What Shrinkage Is—and Is Not

The raw summary is not “wrong”; it answers a different question and does not borrow information through the random-effects distribution. The BLUP can be more stable for predicting random effects under a correctly specified model, especially when a subject has few or noisy observations. But it can also conceal genuine extreme heterogeneity if the assumed random-effects distribution or covariance structure is inappropriate. Shrinkage is not a guarantee of better description in every setting.

A BLUP is a point prediction. It does not, by itself, communicate how uncertain the subject-specific estimate is. In the Gaussian model with known parameters, the conditional covariance of the random effects is

when \(\mathbf{D}\) is invertible. Sparse data generally leave more conditional uncertainty than dense, informative measurements. A plotted predicted line without uncertainty can therefore give a false impression of precision, particularly for subjects with few visits or little time spread.

Common Mistakes

• Calling shrinkage bias in the calculation. A BLUP intentionally combines the subject’s evidence with the random-effects distribution. It is a predictor under the model, not an unshrunk estimate of an independently fixed subject parameter.
• Interpreting the random effect as the total trajectory. The random intercept and slope are departures from the fixed-effects trajectory. The total predicted trajectory combines fixed and random parts.
• Using a raw mean to infer change. Subjects with equal means can have very different time patterns, as the worked example shows.
• Assuming every subject is shrunk by the same amount. The amount depends on the subject’s design and the variance structure; with multiple random effects, correlations and observation times also matter.
• Treating a shrunk line as certain. A point prediction does not show its uncertainty, and strong shrinkage can signal weak subject-specific information rather than proof of a typical trajectory.