Updating Subject-Specific Random Effects

Why a Subject-Specific Update Uses Both Data Streams

In a shared-random-effects joint model, a subject’s unobserved random effects influence both the longitudinal trajectory and the event hazard. Updating those effects therefore requires combining that subject’s biomarker measurements with their event information—not treating either source as a complete update by itself. The purpose here is to derive the conditional posterior kernel for one subject, with all model parameters and the other subjects’ random effects held fixed.

The kernel is the part of a conditional density that depends on the quantity being updated; it is proportional to the full conditional density, but need not integrate to one. This distinction is useful because the longitudinal and survival terms are easy to identify even when their product does not have a familiar named distribution.

Model and assumptions: For subject \(i\), let \(b_i\) be a \(q\)-dimensional random-effects vector with prior \(b_i\sim N_q(0,D)\). Conditional on \(b_i\) and the model parameters, longitudinal measurements are independent with \(y_{ij}\sim N(x_{ij}^{T}\beta+z_{ij}^{T}b_i,\sigma^2)\). The event hazard is \(h_i(t\mid b_i)=h_0(t)\exp\{w_i(t)^T\gamma+\alpha m_i(t;b_i)\}\), where \(m_i(t;b_i)=x_i(t)^T\beta+z_i(t)^Tb_i\). Assume the longitudinal observations and event process are conditionally independent given \(b_i\), covariates, and model parameters; random effects are independent across subjects conditional on \(D\); and censoring is independent and ignorable conditional on the modeled information.

Write the observed event information as \((T_i,\delta_i)\), where \(T_i\) is the observed event or censoring time and \(\delta_i=1\) indicates an event while \(\delta_i=0\) indicates right-censoring. Define the cumulative hazard \(H_i(T_i\mid b_i)=\int_0^{T_i}h_i(u\mid b_i)\,du\). Conditional on the model, the event-process likelihood contribution is \(h_i(T_i\mid b_i)^{\delta_i}\exp\{-H_i(T_i\mid b_i)\}\). Thus censoring contributes the probability of surviving to \(T_i\), while an observed event contributes both survival up to \(T_i\) and the hazard at \(T_i\).

Deriving the Conditional Posterior Kernel

Let \(\theta\) collect the model parameters, including \(\beta,\gamma,\alpha,D,\sigma^2\), and the baseline-hazard parameters. Condition on \(\theta\), all observed data, and the other subjects’ random effects \(b_{-i}\). Because the prior and likelihood factor across subjects conditional on shared parameters, all terms for subjects other than \(i\) are constant with respect to \(b_i\) and cancel from its conditional density.

\begin{aligned} p(b_i\mid \theta,\text{data},b_{-i}) &\propto p(b_i\mid D)\, \prod_{j=1}^{n_i}p(y_{ij}\mid b_i,\theta)\, h_i(T_i\mid b_i)^{\delta_i} \exp\{-H_i(T_i\mid b_i)\}\\ &\propto \exp\left\{-\frac12 b_i^TD^{-1}b_i -\frac{1}{2\sigma^2}\sum_{j=1}^{n_i} \big[y_{ij}-x_{ij}^T\beta-z_{ij}^Tb_i\big]^2\right\}\\ &\qquad{}\times h_i(T_i\mid b_i)^{\delta_i} \exp\{-H_i(T_i\mid b_i)\}. \end{aligned}

The proportionality omits factors that do not depend on \(b_i\), such as Gaussian normalizing constants, when the other quantities are fixed. Taking logarithms gives a convenient conditional log-kernel:

\ell_i(b_i)= -\frac12 b_i^TD^{-1}b_i -\frac{1}{2\sigma^2}\sum_{j=1}^{n_i} \big[y_{ij}-x_{ij}^T\beta-z_{ij}^Tb_i\big]^2 +\delta_i\log h_i(T_i\mid b_i)-H_i(T_i\mid b_i),

up to an additive constant independent of \(b_i\). The first term regularizes the random effects toward their population distribution; the second rewards values that fit the subject’s measurements; the last two terms encode the event-time information. In particular, \(-H_i\) is present for both events and censoring. The hazard-at-the-observed-time term appears only when \(\delta_i=1\).

Key result: The subject-specific conditional posterior kernel is the product of the random-effects prior, the subject’s longitudinal likelihood, and the subject’s event-process likelihood. It is not obtained by updating from the longitudinal measurements alone and then ignoring the event record.

Worked Examples

Worked Example: A Longitudinal Measurement Contribution

Suppose a hypothetical subject has a scalar random intercept \(b_i\sim N(0,0.25)\), known residual variance \(\sigma^2=0.25\), and two measurements. Their fixed-effect means are 1.0 and 1.5, their observed values are 1.2 and 1.4, and each measurement has random-effects coefficient \(z_{ij}=1\). First calculate the prior-times-longitudinal kernel, leaving out the event contribution temporarily.

Ignoring constants, the log-kernel is \(-b_i^2/(2(0.25))-\{(1.2-1.0-b_i)^2+(1.4-1.5-b_i)^2\}/(2(0.25))\). At \(b_i=0\), the prior contribution is 0 and the longitudinal contribution is \(-[(0.2)^2+(-0.1)^2]/0.5=-0.10\). The total is therefore \(-0.10\).

At \(b_i=0.2\), the prior contribution is \(-0.2^2/0.5=-0.08\). The residuals are \(1.2-(1.0+0.2)=0\) and \(1.4-(1.5+0.2)=-0.3\), so the longitudinal contribution is \(-[0^2+(-0.3)^2]/0.5=-0.18\). The total log-kernel is \(-0.26\), a difference of \(-0.26-(-0.10)=-0.16\) relative to \(b_i=0\). The prior-times-longitudinal kernel at \(b_i=0.2\) is thus \(\exp(-0.16)\approx0.8521\) times its value at zero. This is not yet the joint-model comparison because the event contribution has not been included.

Worked Example: An Observed Event Contributes a Hazard and Survival Term

For a hypothetical event record, let \(h_i(t\mid b_i)=0.2\exp(0.5b_i)\) per year, with constant hazard over the observed two years. Suppose an event is observed at \(T_i=2\) years, so \(\delta_i=1\). The event log-likelihood contribution, up to constants, is \(\log\{0.2\exp(0.5b_i)\}-0.4\exp(0.5b_i)\).

Compare \(b_i=0.3\) with \(b_i=0\). The hazard-at-event log-term difference is \(0.5(0.3)=0.15\). The cumulative-hazard difference is \(0.4\{\exp(0.15)-1\}\), where \(\exp(0.15)\approx1.1618342\). Therefore the event log-likelihood difference is \(0.15-0.4(1.1618342-1)=0.15-0.0647337=0.0852663\). The event likelihood at \(b_i=0.3\) is \(\exp(0.0852663)\approx1.0890\) times its value at zero. This event contribution alone favors \(b_i=0.3\) over zero, but it does not determine the posterior comparison: the prior and measurements also matter.

If the same subject were censored at two years instead, \(\delta_i=0\) and the hazard-at-time term would disappear. The log-likelihood difference would be only \(-0.0647337\), giving a ratio \(\exp(-0.0647337)\approx0.9373\). In this example the censoring record favors the lower hazard value, illustrating why censoring is informative about the random effects through survival to the censoring time.

Worked Example: Combining the Prior, Measurements, and Event Record

Use the measurements and prior from the first example, and suppose the event model and observed event are as in the second example. Compare the full conditional kernels at \(b_i=0.3\) and \(b_i=0\). For the longitudinal term at \(b_i=0.3\), the residuals are \(1.2-(1.0+0.3)=-0.1\) and \(1.4-(1.5+0.3)=-0.4\). Its log contribution is \(-[(-0.1)^2+(-0.4)^2]/0.5=-0.34\); the prior log contribution is \(-0.3^2/0.5=-0.18\).

At \(b_i=0\), the corresponding longitudinal contribution is \(-0.10\) and the prior contribution is 0. The longitudinal-plus-prior difference is therefore \((-0.34-0.18)-(-0.10)=-0.42\). From the preceding calculation, the event log-likelihood difference is \(0.15-0.4\{\exp(0.15)-1\}\approx0.0852663\). Adding the components gives the full log-kernel difference: \(-0.42+0.0852663=-0.3347337\). Thus the full conditional kernel at \(b_i=0.3\) is \(\exp(-0.3347337)\approx0.7156\) times its value at zero. The event record alone favored 0.3, but the prior and longitudinal data together outweigh that contribution in this comparison. This is a comparison of kernel values, not a posterior probability that \(b_i\) equals either candidate value.

Why the Conditional Density Is Usually Not Gaussian

If only the Gaussian prior and Gaussian longitudinal likelihood were present, their product would yield a Gaussian conditional distribution for \(b_i\). The event contribution generally changes that. In the hazard model above, the log-hazard at \(T_i\) depends on \(b_i\), and the cumulative hazard integrates an exponential function of \(b_i\) over time. Consequently, even though the longitudinal part is quadratic in \(b_i\), the full log-kernel need not be quadratic. A Gaussian update based only on the longitudinal data would therefore target the wrong conditional distribution whenever the event likelihood depends on \(b_i\).

The full conditional kernel is the target to evaluate for this subject, conditional on the current values of shared parameters. This is distinct from a marginal likelihood contribution, which integrates over the random effects, and from an update of a shared association parameter. In practice, the conditional density may require a numerical sampling strategy; the derivation here specifies its target rather than a particular sampler.

Diagnostics, Failure Modes, and Limitations

A useful diagnostic is to inspect the three contributions to the log-kernel separately as \(b_i\) varies: the prior, longitudinal fit, and event-process fit. If the longitudinal-only calculation looks plausible but the joint update changes substantially, determine whether the event record justifies that change. For an observed event, both the event-time hazard and cumulative hazard must be present; for censoring, only the cumulative-hazard survival term remains. Compare the implementation at selected candidate values against direct substitution into the displayed formula.

Dropping the cumulative hazard. Using only the hazard at an observed event ignores the probability of surviving until that event. Include \(-H_i(T_i\mid b_i)\) for events and censored observations alike.
Adding an event-time hazard for censoring. A censoring time is not an observed event. Set \(\delta_i=0\), so there is no \(\log h_i(T_i\mid b_i)\) term.
Using the wrong sign. The cumulative hazard enters as \(-H_i\), not \(+H_i\). A larger cumulative hazard means less probability of remaining event-free to \(T_i\).
Comparing rounded totals as if exact. When reporting a small log-kernel difference, calculate it from unrounded component values and round only the final result. Otherwise the displayed subtraction may not support the stated digits.

The conditional target also relies on the model assumptions: the chosen shared-effects structure must describe how \(b_i\) enters the trajectory and hazard, the conditional independence assumptions must be defensible, and the censoring mechanism must be ignorable as specified. If event dependence remains after conditioning on \(b_i\) and modeled covariates, or if the hazard link is misspecified, this kernel faithfully represents the stated model but may not adequately represent the data-generating process. With sparse measurements or few events for a subject, the conditional distribution may remain broad and sensitive to the population-level prior \(D\).

Key takeaway: Updating one subject’s shared random effects means multiplying three pieces: the random-effects prior, that subject’s longitudinal likelihood, and that subject’s event likelihood. The event contribution is \(\delta_i\log h_i(T_i\mid b_i)-H_i(T_i\mid b_i)\) on the log scale.

Check Your Understanding

Try each question before reading the solutions below.

For a right-censored subject, which terms from the event-process log-likelihood depend on \(b_i\)?
In the first worked example, suppose the event information is omitted. Find the conditional distribution of \(b_i\) given the two measurements and the prior.
For the event model in the second example, what is the event-likelihood ratio at \(b_i=0.3\) versus \(b_i=0\) if the subject is censored at two years?
Why can the longitudinal-only Gaussian conditional distribution not generally be used as the full joint-model conditional distribution?

Solutions. (1) Only \(-H_i(T_i\mid b_i)\) remains; the factor \(\delta_i\log h_i(T_i\mid b_i)\) is zero because \(\delta_i=0\). (2) The prior precision is \(1/0.25=4\), and each measurement contributes precision \(1/0.25=4\), so the posterior precision is \(4+4+4=12\) and the variance is \(1/12\). The measurement residuals relative to their fixed means are \(0.2\) and \(-0.1\), giving posterior mean \(\{(0.2-0.1)/0.25\}/12=0.4/12=0.03333\). Thus \(b_i\mid y_i\sim N(0.03333,1/12)\) under this longitudinal-only calculation. (3) The log-likelihood difference is \(-0.4\{\exp(0.15)-1\}\approx-0.0647337\), so the ratio is \(\exp(-0.0647337)\approx0.9373\). (4) The event likelihood generally adds a nonquadratic function of \(b_i\), through the hazard and cumulative hazard. The Gaussian conditional from the prior and measurements alone therefore omits part of the joint-model target.