Frailty Models For Unobserved Heterogeneity

Why Model Unobserved Heterogeneity?

People with the same measured covariates may still differ in their underlying event risk. In a duration model, an unmeasured characteristic can make one person more susceptible to an event throughout follow-up. A model that ignores these persistent differences may describe the observed population hazard poorly, even when each person's hazard follows a simple pattern.

A frailty model represents such latent susceptibility by multiplying the hazard by a positive random variable. Its implications depend on whether the latent effect belongs to one person or is shared by members of a group. Individual frailty captures unobserved differences among people; shared frailty can also induce dependence among event times within a cluster, such as patients treated at the same hypothetical clinic.

The frailty is unobserved, not an additional measured covariate. Conditional on its value, the hazard model describes an individual's event process. Marginal quantities average over the frailty distribution. These are different estimands: a conditional hazard compares individuals with the same frailty, whereas a marginal hazard describes the changing composition of the population still at risk.

Gamma Frailty and Marginal Survival

A common choice is a gamma frailty with mean one and variance \(\theta\), where \(\theta>0\) measures unobserved heterogeneity. Use the shape-scale parameterization: \(Z\sim\operatorname{Gamma}(1/\theta,\theta)\). Its mean is \((1/\theta)\theta=1\), and its variance is \((1/\theta)\theta^2=\theta\). The limiting case \(\theta=0\) represents no frailty, with \(Z=1\) with probability one.

This expression follows by averaging the conditional survival over \(Z\). The gamma Laplace transform is \(E[\exp(-sZ)]=(1+\theta s)^{-1/\theta}\) for \(s\geq 0\). Setting \(s=A(t,x)\) gives the displayed result. The formula also approaches \(\exp[-A(t,x)]\) as \(\theta\) tends to zero, recovering the model without frailty.

Marginal Hazards and Risk-Set Selection

Differentiating the marginal cumulative hazard \(-\log S_m(t\mid x)\) gives the marginal hazard. Write \(q=\exp(x^\top\beta)\). Since \(A(t,x)=qH_0(t)\), its derivative is \(q h_0(t)\). Therefore,

The denominator shows why a marginal hazard need not preserve the conditional proportional-hazards structure. At a fixed time, individuals with larger frailty are more likely to have experienced the event already. The people remaining at risk are consequently selected toward lower frailty, so their average latent susceptibility can decline over time.

This also makes the selection mechanism explicit: the relevant mean frailty among survivors falls as cumulative hazard \(A(t,x)\) grows. The conditional hazard for someone whose frailty is \(Z\) remains \(Z h_0(t)\exp(x^\top\beta)\); it is the population's risk-set composition that changes.

Frailty can also make marginal hazard ratios vary with time. For a one-unit covariate contrast with conditional hazard multiplier \(r=\exp(\beta)\), the marginal hazard ratio, relative to the covariate value with multiplier one, is

In the simple comparison where the reference multiplier is one, this ratio is \(r\{1+\theta H_0(t)\}/\{1+\theta H_0(t)r\}\). It equals the conditional ratio \(r\) at the start, when \(H_0(t)=0\), but for \(r>1\) it moves toward one as cumulative hazard grows. Thus, a constant conditional effect does not generally imply a constant marginal hazard ratio.

Individual and Shared Frailty

With individual frailty, each person has a latent effect \(Z_i\). A common assumption is that the \(Z_i\) are independent draws from a specified distribution, independent of measured covariates. This describes unobserved heterogeneity across people and produces a marginal hazard that may vary with duration. Independent individual frailties do not, by themselves, create dependence between different people's event times.

With shared frailty, all members of a cluster share a latent effect \(Z_j\). Conditional on that shared value and the covariates, event times are often assumed independent. Marginally, they are dependent because a high value of \(Z_j\) raises the hazard for every member, while a low value lowers it for every member. The cluster could represent a family, workplace, or other group with common unobserved exposure or susceptibility.

For a shared gamma frailty and conditionally independent event times, the same calculation extends to \(m\) people: joint survival is \(\{1+\theta\sum_{i=1}^m A_i(t,x_i)\}^{-1/\theta}\). This expression is not the product of their marginal survival probabilities. The resulting dependence matters for likelihood construction and uncertainty estimation: members of a cluster cannot generally be treated as independent after the shared frailty has been integrated out.

Assumptions, Interpretation, and Pitfalls

The gamma formulas rely on a particular model, not merely on the existence of unobserved differences. They assume a positive multiplicative frailty, a specified gamma distribution, and conditional survival of the proportional-hazards form. The mean-one convention separates the overall hazard scale from the frailty multiplier; without a scale constraint, the baseline hazard and frailty scale would not be separately identified. The assumption that frailty is independent of covariates is also substantive. If latent susceptibility is associated with measured covariates, the simple marginal formulas above do not automatically apply.

In applications, the frailty variance \(\theta\) is estimated as part of a model, and conclusions depend on the assumed frailty distribution and cluster structure. A shared frailty is appropriate only when the scientific setting supports a common latent factor within the chosen clusters. An individual frailty model answers a different question and does not represent within-cluster dependence.

• Mistake: treating the marginal hazard as the conditional hazard at mean frailty. Substituting \(Z=E(Z)=1\) ignores how survival changes the frailty distribution among those still at risk. Use the marginal survival or marginal hazard formula when the target is population-level.
• Mistake: interpreting a conditional hazard ratio as the marginal hazard ratio. Under gamma frailty, marginal hazard ratios can vary with time even when the conditional coefficient is constant. State which comparison the model estimates.
• Mistake: assuming shared frailty is just another variance adjustment. It induces dependence among cluster members after the latent effect is averaged out. Account for that dependence rather than treating all event times as marginally independent.