Worked Comparison of Shared and Individual Frailty Fits

Why Compare Frailty Structures?

For recurrent events in clusters, dependence may arise because people in the same cluster share unmeasured risk, because some individuals have persistently higher event rates than others, or because both mechanisms operate. A shared frailty represents cluster-level heterogeneity; an individual frailty represents additional person-level heterogeneity. Choosing between them matters: the models can yield different likelihoods, variance estimates, and predictions even when their average event rates are similar.

The comparison is meaningful only when the candidate models use the same observations, follow-up definition, covariates, baseline specification, and likelihood convention. A larger maximized likelihood is not by itself evidence that a more complex model is preferable; it will generally be at least as large when the simpler model is nested within it. Information criteria and scientific plausibility also matter.

For clarity, consider a piecewise-constant Poisson rate model. Conditional on the frailties, counts are independent Poisson variables with means \(e_{ij}\lambda\exp(x_{ij}^{T}\beta)Z_jW_{ij}\). Here \(\lambda\) is a baseline rate, \(x_{ij}\) is a covariate vector, and \(\beta\) is its coefficient vector. Set \(Z_j=1\) in the individual-only model and \(W_{ij}=1\) in the shared-only model. The no-frailty model sets both to 1. In applications with time-varying rates or recurrent-event histories, the corresponding conditional event-process likelihood replaces this simple count likelihood; the essential comparison still requires the same observation model across fits.

The basic assumptions are that the specified conditional event model is adequate, that the frailty distribution and independence structure are reasonable, and that censoring and observation processes are handled appropriately. In particular, an exposure offset must reflect actual at-risk time, and the model must not treat an unobserved period as event-free follow-up. These assumptions should be assessed rather than inferred merely from a successful optimizer.

What the Likelihood Comparison Measures

In the shared-only model, conditional independence given \(Z_j\) does not imply marginal independence. Integrating over the common frailty couples all counts in a cluster. Write \(\mu_{ij}=e_{ij}\lambda\exp(x_{ij}^{T}\beta)\), \(N_j=\sum_iY_{ij}\), and \(M_j=\sum_i\mu_{ij}\). Integrating the gamma density gives the cluster likelihood contribution

In the individual-only model, each person’s count has its own gamma mixture, so the marginal likelihood factors across individuals. In the combined model, integrating \(Z_j\) and each \(W_{ij}\) does not generally reduce to a product of two simple negative-binomial likelihoods: conditional on \(Z_j\), the individual frailty mixture still depends on \(Z_j\). The combined likelihood should therefore be evaluated by the appropriate integration or likelihood algorithm, not by multiplying the shared-only and individual-only likelihoods.

Compare Maximized Fits on the Same Data

Each candidate should be fitted by maximizing its own marginal likelihood, including the frailty integration. Use the same event records and covariates, and make sure that all likelihoods include the same constants and treatment of censoring. For a nested comparison, the larger model’s maximized likelihood cannot be smaller than that of the smaller model when both are optimized over the same parameter space and the smaller model is obtained by setting a variance to zero.

The following hypothetical output is for one dataset of 40 clusters. Each model contains the same two baseline and covariate parameters; the variance parameters are additional. The listed values are assumed to be maximized log likelihoods from a common likelihood convention.

Why the Variance Components Change Predictions

Frailty variances affect the distribution of future counts, not their mean under this mean-one parameterization. For example, for \(m\) individuals in a new cluster with equal conditional mean \(H\) per person, the expected total count is \(mH\) in all three models. But the variance differs because the frailties induce different dependence patterns.

For shared-only frailty, the total count conditional on \(Z\) is Poisson with mean \(mHZ\). The law of total variance gives \(\operatorname{Var}(N)=mH+(mH)^2\theta_S\). Under individual-only frailty the counts are independent marginally, each with variance \(H+H^2\theta_I\), so \(\operatorname{Var}(N)=mH+mH^2\theta_I\). In the combined model, the random total intensity is \(HZ\sum_iW_i\). Since \(E(Z^2)=1+\theta_S\), \(E(W_i)=1\), and \(E[(\sum_iW_i)^2]=m^2+m\theta_I\), its variance is

Justification and Practical Diagnostics

The variance formulas follow directly from conditional variance. For the combined model, write \(N\mid Z,W\sim\operatorname{Poisson}(HZ\sum_iW_i)\). Therefore, \(\operatorname{Var}(N)=E\{\operatorname{Var}(N\mid Z,W)\}+\operatorname{Var}\{E(N\mid Z,W)\}\). The first term is \(mH\), because \(E(Z\sum_iW_i)=m\). For the second term, independence yields \(E[(Z\sum_iW_i)^2]=(1+\theta_S)(m^2+m\theta_I)\); subtracting its squared mean \(m^2\) gives \(m\theta_I+\theta_S(m^2+m\theta_I)\). This establishes the displayed result and shows why simply adding the two variance estimates would be wrong.

A comparison should include checks that can reveal a poor fit or a numerical problem:

• Check optimizer convergence, gradient size, and whether estimates change with reasonable starting values. A variance estimate near a boundary may signal weak information, not a stable component.
• Compare observed and model-predicted cluster-level event-count distributions, including the number of zero-event clusters and the upper tail. A model can match the overall mean while missing clustering or extreme counts.
• Inspect residual patterns against follow-up time, covariates, cluster size, and exposure. Systematic patterns can indicate a misspecified rate, time trend, or observation process rather than the wrong frailty level alone.
• Check whether conclusions are driven by a few large clusters or by unequal cluster sizes. Refit under plausible baseline and frailty-distribution choices when the variance estimates are sensitive.
• Verify that conditional event assumptions are plausible. Frailty does not automatically correct event-history dependence, changing risk after prior events, or informative dropout.

A shared-only and an individual-only model are generally not nested within one another. Their likelihood difference therefore does not have the usual simple likelihood-ratio test interpretation. The combined model contains either simpler model as a boundary case when the corresponding variance is zero, but testing a zero variance is also a boundary problem; ordinary chi-square reference distributions need not apply. The earlier material on testing zero frailty variance at the boundary is relevant when formal testing is required. AIC or a carefully justified boundary-aware procedure is preferable to an unqualified ordinary likelihood-ratio test.

Information criteria are useful but have limits. AIC ranks candidates only relative to the set considered and does not establish causal interpretation, correct specification, or transportability. Variance estimates can be weakly identified, particularly when few clusters are observed or follow-up is limited. Shared and individual components may trade off, so uncertainty in both estimates and predictions should be considered. Sensitivity to the assumed frailty distribution also matters: the earlier material on sensitivity to frailty distribution addresses how conclusions can change when the gamma assumption is altered.

Common Mistakes and Model Choice

A frequent mistake is to say that a combined fit “proves” both types of heterogeneity because its log likelihood is largest. The log likelihood is expected to increase as parameters are added; compare the gain with complexity and assess fit, uncertainty, and scientific plausibility. Another mistake is to interpret \(\hat\theta_S\) from a shared-only fit as directly comparable to \(\hat\theta_S\) from a combined fit. The estimates answer different conditional questions because the models allocate heterogeneity differently.

The main alternatives have distinct uses. Shared-only models are parsimonious when cluster-level dependence is the target and individual heterogeneity is not needed. Individual-only models allow persistent person-to-person variation but do not induce dependence between distinct individuals in a cluster. Combined models can represent both sources, but require more information and may be unstable in small datasets. Cluster-robust standard errors, covered in the earlier material on frailty versus cluster-robust standard errors, adjust uncertainty for dependence without specifying a frailty distribution; they do not provide the same latent-risk structure or frailty-based predictions.