Nested Frailty for Multilevel Clustering

Why Model More Than One Clustering Level?

Event observations can be related for more than one reason. Recurrent events or multiple records from the same unit may share unit-level risk, while units within a hospital, clinic, or other institution may also share broader conditions. A single shared frailty for each unit captures the first source of heterogeneity but not the additional dependence associated with a common institution. A nested frailty model represents both levels in one hierarchy.

The model below is multiplicative: an institution-level frailty affects all its units, and a unit-level frailty affects all observations within that unit. The resulting covariance has a recognizable structure: observations in the same unit share both components; observations in different units at the same institution share only the institution component; observations in different institutions share neither.

Model and Assumptions

Definition: Let \(g\) index institutions, \(j\) index units within institution \(g\), and \(k\) index event observations or event processes within unit \(j\). Let \(U_g\) be the institution frailty and \(V_{gj}\) the unit frailty. Define the event multiplier for unit \(gj\) as \(W_{gj}=U_gV_{gj}\). Assume the frailties are positive, have mean one, and satisfy \(\operatorname{Var}(U_g)=\theta_U\) and \(\operatorname{Var}(V_{gj})=\theta_V\). Frailties are independent across institutions, and the unit frailties are mutually independent and independent of all institution frailties. Conditional on the frailties and covariates, the event processes are independent.

A mean of one makes each frailty a relative risk multiplier: the hierarchy adds heterogeneity without changing the average multiplier at either level. The two variance parameters are nonnegative. The independence assumptions are substantive, not just notation: they say, for example, that after accounting for the institution frailty, different units have independent unit frailties.

For a recurrent-event process with covariate vector \(x_{gjk}(t)\), a common conditional intensity specification is

\lambda_{gjk}(t\mid U_g,V_{gj}) =U_gV_{gj}\lambda_0(t)\exp\{x_{gjk}(t)^\mathsf{T}\beta\}.

Here \(\lambda_0(t)\) is the baseline intensity and \(\beta\) is the covariate coefficient vector. This is a conditional model: once the two frailties and covariates are specified, the intensity is determined. Conditional independence means that the event processes do not have further dependence after conditioning on those quantities. It does not assert marginal independence.

Deriving the Multiplier Covariance Structure

The variance and covariance of the multipliers follow directly from independence and the mean-one assumptions. For a single unit,

\begin{aligned} \operatorname{Var}(W_{gj}) &=E(U_g^2)E(V_{gj}^2)-1\\ &=(1+\theta_U)(1+\theta_V)-1\\ &=\theta_U+\theta_V+\theta_U\theta_V. \end{aligned}

For two distinct units \(j\ne j'\) in the same institution, independence of their unit frailties gives

\begin{aligned} \operatorname{Cov}(W_{gj},W_{gj'}) &=E(U_g^2V_{gj}V_{gj'})-E(W_{gj})E(W_{gj'})\\ &=E(U_g^2)E(V_{gj})E(V_{gj'})-1\\ &=(1+\theta_U)-1=\theta_U. \end{aligned}

For units in different institutions, all frailties in their multipliers are independent, so their covariance is zero. Thus the shared institution contributes covariance across units, while the unit frailty adds variance only for observations sharing a unit.

Result: The multiplier variance is \(\theta_U+\theta_V+\theta_U\theta_V\) within a unit. The multiplier covariance is \(\theta_U\) for distinct units at the same institution, and zero for different institutions. If the multiplier variance is positive, the correlation for two observations sharing a unit is one because they have the same multiplier. If the multiplier variance is zero, that correlation is undefined. For distinct units at the same institution, the correlation is \(\theta_U/(\theta_U+\theta_V+\theta_U\theta_V)\) when the denominator is positive.

The correlation formula is not defined when both variance components are zero: every multiplier is then the constant one, so there is no random variation with which to define a correlation. More generally, a correlation is undefined whenever either variable has zero variance. A zero covariance between distinct institutions corresponds to zero correlation only when both multipliers have positive variance.

Worked Example: Comparing the Three Pair Types

Suppose hypothetical institution and unit frailties have mean one, with \(\theta_U=0.20\) and \(\theta_V=0.50\). Calculate the multiplier variance and the correlation for distinct units in the same institution.

The variance for one unit is

\operatorname{Var}(W_{gj}) =0.20+0.50+(0.20)(0.50)=0.80.

Distinct units at the same institution have covariance \(0.20\), so their multiplier correlation is

\operatorname{Corr}(W_{gj},W_{gj'}) =\frac{0.20}{0.80}=0.250.

Two observations in the same unit share the multiplier and have correlation one, since its variance is positive. Two observations in different institutions have covariance zero and, here, correlation zero because the multiplier variance is positive. These are correlations of the latent multipliers, not automatically correlations of event times or event indicators.

From Frailty Covariance to Recurrent-Event Counts

The multiplier covariance has a direct implication for counts under a specified observation model. Consider counts \(N_a\) and \(N_b\) from two event processes observed over fixed intervals. Let \(A_a\) and \(A_b\) be their respective integrated baseline intensities, including any fixed covariate multiplier over those intervals. Suppose the counts are conditionally independent Poisson variables with means \(W_aA_a\) and \(W_bA_b\), given the frailties. The law of total covariance gives

\begin{aligned} \operatorname{Cov}(N_a,N_b) &=E\{\operatorname{Cov}(N_a,N_b\mid\text{frailties})\}\\ &\quad+\operatorname{Cov}\{E(N_a\mid\text{frailties}),E(N_b\mid\text{frailties})\}\\ &=0+A_aA_b\operatorname{Cov}(W_a,W_b). \end{aligned}

The first term is zero because of conditional independence; the second is the covariance induced by the shared frailties. For a single count, the law of total variance also gives \(\operatorname{Var}(N_a)=A_a+A_a^2\operatorname{Var}(W_a)\). These formulas require the conditional Poisson model and conditional independence; they are not general formulas for arbitrary event outcomes.

Worked Example: Count Covariances at Each Level

Continue with \(\theta_U=0.20\) and \(\theta_V=0.50\). Suppose two hypothetical event processes each have integrated baseline intensity 2 over their observation interval. Calculate their count covariance if they are in the same unit, in different units at the same institution, or in different institutions.

For the same unit, the multiplier covariance equals its variance, \(0.80\). Therefore,

\operatorname{Cov}(N_a,N_b)=2(2)(0.80)=3.20

in squared count units. For different units at the same institution, the multiplier covariance is \(0.20\), giving

\operatorname{Cov}(N_a,N_b)=2(2)(0.20)=0.800.

For different institutions, the multiplier covariance is zero, so the count covariance is \(2(2)(0)=0\). The calculation assumes conditional Poisson counts and conditional independence. If those assumptions fail, these count covariance results need not hold even if the marginal frailty hierarchy remains a useful description.

Event Indicators Are Not Frailty Multipliers

A shared multiplier does not mean that two binary outcomes have the same correlation as the multiplier. For example, conditional on a multiplier \(W\), the probability of at least one event by time \(t\) for cumulative baseline intensity \(A(t)\) is \(q(W)=1-\exp\{-WA(t)\}\). Marginal outcome covariance depends on the variability of this nonlinear probability, not simply on \(\operatorname{Var}(W)\). For distinct conditionally independent outcomes, their covariance is the covariance of their conditional event probabilities.

Worked Example: Same-Institution Event Indicators

For an easily checked hypothetical illustration, let each institution frailty independently equal \(0.5\) or \(1.5\), each with probability \(1/2\), and let each unit frailty independently have the same two-point distribution. These are positive, mean-one frailties with variance \(0.25\) at each level. Consider event indicators by a fixed time, with \(A(t)=0.40\) for both observations. For distinct units at the same institution, calculate the covariance of the indicators.

Given \(U=u\), the two unit frailties and event outcomes are independent. The conditional event probability averaged over a unit frailty is

m(u)=\tfrac12\{1-e^{-0.40u(0.5)}\} +\tfrac12\{1-e^{-0.40u(1.5)}\}.

At \(u=0.5\), this is \(\tfrac12(1-e^{-0.10})+\tfrac12(1-e^{-0.30})=0.1772\). At \(u=1.5\), it is \(\tfrac12(1-e^{-0.30})+\tfrac12(1-e^{-0.90})=0.4263\). The institution frailty takes these two values with equal probability, so the covariance of the distinct-unit indicators is the variance of \(m(U)\):

\operatorname{Cov}(I_1,I_2) =\frac{(0.4263-0.1772)^2}{4} \approx 0.01552.

The covariance is positive because the shared institution frailty raises both conditional event probabilities. It is not the multiplier covariance \(0.25\): the event probabilities are bounded nonlinear functions of the multiplier. This calculation also relies on conditional independence of the two outcomes given the frailties.

Assumptions, Diagnostics, and Limitations

A nested frailty model is most useful when the hierarchy reflects how observations were generated and when each level has enough replication to inform its variance. Before interpreting fitted components, check that institutions and units are correctly identified, that units belong to only one institution in the modeled hierarchy, and that the conditional event model is plausible. The model assumes the chosen frailty distribution, mean-one scaling, and independence between levels; a nested structure alone does not verify these assumptions.

Check the grouping structure. If units are assigned to the wrong institution or substantial cross-institution sharing exists, the implied covariance pattern is incorrect.
Check for remaining dependence. Dependence among processes after conditioning on modeled covariates and frailties, or systematic time patterns not represented by the baseline intensity, can undermine the conditional independence assumption.
Assess information at each level. Few institutions or little replication within institutions can make it difficult to distinguish \(\theta_U\) from \(\theta_V\). A variance estimate near zero is not, by itself, proof that the corresponding level has no heterogeneity.
Interpret the components conditionally. The model describes latent heterogeneity under its assumptions; it does not establish that an institution or unit characteristic causes the observed differences.

The multiplicative specification has a particular consequence: institution and unit effects combine on the intensity scale, and their variances interact through the term \(\theta_U\theta_V\). The covariance formulas above do not depend on choosing gamma or lognormal frailties, but marginal survival probabilities and likelihood integrations generally do depend on the frailty distributions. Those distributions can therefore matter even when their means and variances agree.

Common Mistake and Model Comparisons

Common mistake: “All observations at one unit have correlation one, so the observed event outcomes must be identical.” The correction is that they share the same latent multiplier, not necessarily the same event outcome. Conditional event processes can still differ randomly. Moreover, the multiplier's correlation with itself is one only when its variance is positive; if both frailty variances are zero, the multiplier is constant and that correlation is undefined.

A single shared unit frailty is a simpler alternative when institution-level variation is not part of the scientific structure. It cannot, by itself, represent the extra covariance between distinct units at one institution. At the other extreme, institution-only frailty gives those distinct units shared risk but omits additional common heterogeneity among observations within a unit. A nested model separates these two covariance contributions, but adds a variance component and therefore requires sufficient information at both levels. Cluster-robust standard errors address uncertainty under specified forms of clustering; they do not introduce the latent hierarchy or estimate these frailty variance components.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

If \(\theta_U=0.30\) and \(\theta_V=0.20\), what is the variance of a unit multiplier and the correlation for distinct units at the same institution?
Under conditional Poisson counts, two distinct units at one institution have \(A_a=1.5\), \(A_b=2\), and \(\theta_U=0.30\). What is their count covariance?
What is the multiplier covariance for units at different institutions, and what independence assumption supports that result?
Why is the correlation of a unit multiplier with itself undefined when \(\theta_U=\theta_V=0\)?

Solutions. (1) The variance is \(0.30+0.20+(0.30)(0.20)=0.56\); the distinct-unit correlation is \(0.30/0.56=0.536\) to three significant figures. (2) The multiplier covariance is \(0.30\), so the count covariance is \((1.5)(2)(0.30)=0.900\) squared count units. (3) It is zero because the institution frailties are independent across institutions and the unit frailties are independent across units and of the institution frailties. (4) Both components then have zero variance, so the multiplier is the constant one; correlation requires positive variance in both variables.