Chain Diagnostics with Rare Events

Why Rare Events Change What to Inspect

In a Bayesian joint model, a small number of observed events can leave survival-related parameters weakly informed by the likelihood. The resulting posterior may be broad, strongly shaped by the prior, or difficult for an algorithm to explore. These outcomes are not equivalent: a broad posterior can be sampled well, while a seemingly narrow estimate can still be poorly sampled. Diagnostics must therefore assess how the chains behave, not just whether the final estimate looks plausible.

Consider a shared-random-effects model with longitudinal measurements \(y_{ij}\) for subject \(i\) at visit \(j\), subject-specific random effects \(b_i\), and an event hazard \(h_i(t\mid b_i)=h_0(t)\exp\{\alpha m_i(t)\}\), where \(m_i(t)\) is the model-based longitudinal value at time \(t\). The parameter \(\alpha\) describes the association between that longitudinal value and the event hazard. The baseline hazard \(h_0(t)\) describes the hazard when the association contribution is zero. A variance component, such as \(\tau^2\), describes variation in the random effects. With few events, the event likelihood may provide little information about \(\alpha\) and the baseline hazard, and can also affect inference about \(\tau^2\) through the shared random effects.

The aim here is targeted chain assessment for these parameters. A trace plot displays draws in their sampling order, with chain identity retained. Rhat compares variation within chains with variation between chains. Effective sample size (ESS) summarizes how much information autocorrelated draws provide about a posterior quantity. These diagnostics answer different questions, so none should be used alone.

Conditions for Interpreting Diagnostics

Diagnostics are meaningful only in relation to a correctly specified sampling target. The chains must be sampling the same posterior distribution, and retained draws should come from the post-warmup portion after the algorithm has adapted. The chains should start from suitably dispersed initial states; chains initialized in nearly identical locations can agree while all missing an important posterior region. For the usual convergence interpretations, the chains should be stationary and explore the relevant posterior support. Rhat and ESS cannot establish these conditions with certainty.

Sparse events make parameter-specific checking especially important. Do not infer that every parameter mixes poorly merely because the event count is low. Nor should good diagnostics for the longitudinal regression coefficients be taken as evidence that the association parameter or hazard parameters are well sampled. Inspect the quantities that the event likelihood informs, including the association parameter, relevant baseline-hazard parameters, and random-effects variance components.

Before interpreting a diagnostic, check that plotted draws are post-warmup, chains are distinguishable by color or line type, and all chains use the same parameterization and scale. For a positive variance parameter, inspect the scale used in the model as well as any transformed quantity sampled by the algorithm. A trace of \(\log(\tau)\), for instance, is not on the same scale as a trace of \(\tau\).

What Rhat and ESS Measure

A basic derivation explains the role of Rhat. Suppose \(m\) chains each have \(n\) retained draws of a scalar parameter. Let \(W\) be the average of the within-chain sample variances, and let \(B\) be \(n\) times the sample variance of the chain means. An estimate of the marginal variance combines within-chain and between-chain variation:

If chains explore the same distribution, their means and variances should become similar, so this ratio should approach 1. If chain means differ, \(B\) can be large relative to \(W\), raising Rhat. The formula is a useful explanation, not the full modern diagnostic: practical software commonly reports split, rank-normalized Rhat, which also helps detect certain non-stationary behavior and differences in distributional shape. A value below 1.01 is often used as a screening guideline, not as proof of convergence.

ESS accounts for autocorrelation. If \(M\) is the total number of retained draws across chains and \(\rho_k\) is the lag-\(k\) autocorrelation, a standard approximation is

Positive autocorrelation increases the denominator and reduces ESS. This follows because the variance of an average of correlated draws includes covariance terms: positive dependence makes the average less precise than an average of the same number of independent draws. Software estimates ESS using more careful methods than simply summing an infinite sequence. Bulk ESS concerns central posterior behavior; tail ESS assesses estimation in the distribution's tails. Monte Carlo standard error should be judged against the precision needed for the reported result, rather than against a universal ESS cutoff.

Read the Traces in the Model's Terms

A trace is useful when interpreted against the parameter's meaning and the event information. For \(\alpha\), ask whether chains overlap in their ranges, move repeatedly through the shared range, and show comparable behavior over time. A trace that looks like a dense horizontal band is not automatically reassuring: a chain can remain trapped in one region while appearing noisy at that scale. View chains separately as well as overlaid, and consider the range or scale that matters scientifically.

For baseline hazards, inspect parameters for each time interval or basis component that is informed by events. A parameter for a late interval with little follow-up or no events may be especially weakly informed. In a piecewise exponential model with hazard \(\lambda_k\) in interval \(k\), event count \(d_k\), and total at-risk exposure \(E_k\), the hazard-dependent likelihood contribution is proportional to \(\lambda_k^{d_k}\exp(-\lambda_k E_k)\). When \(d_k=0\), the contribution is \(\exp(-\lambda_k E_k)\), not a point mass at zero hazard. The trace may still have a broad posterior, particularly when exposure is also small.

For a variance component, inspect whether draws explore its full plausible range, including values close to zero when those are supported. A long, slowly moving trace can yield a low ESS even if chain means and Rhat look acceptable. When a variance parameter has a skewed or heavy-tailed posterior, check tail ESS and Monte Carlo error for tail summaries in addition to the trace and central ESS.

ESS, Apparent Stability, and Practical Precision

A stable-looking trace can still have high autocorrelation. For intuition, if a stationary sequence has approximately geometric autocorrelation \(\rho_k=\rho^k\) with lag-one correlation \(\rho\), then \(\sum_{k=1}^{\infty}\rho^k=\rho/(1-\rho)\). Substituting into the ESS expression gives \(N_{\mathrm{eff}}\approx M(1-\rho)/(1+\rho)\). This is an approximation for illustration; real chains need not have geometric autocorrelation.

Use Diagnostics Together, and Know Their Limits

The main alternatives are not competing tests but complementary views of sampling behavior. Trace plots reveal time patterns and chain-specific sticking. Rhat summarizes between-chain agreement relative to within-chain variation. ESS summarizes autocorrelation-related information loss and can be translated into Monte Carlo precision. Autocorrelation plots help locate slow decay in dependence. For Hamiltonian Monte Carlo, divergences are a separate warning about problematic exploration of posterior geometry; a favorable Rhat does not make divergences irrelevant.

A common mistake is to treat few events as a reason to ignore a high Rhat: “The posterior is uncertain anyway, so chain disagreement is expected.” The correction is that uncertainty and poor mixing are distinct. A diffuse posterior can be sampled well, and chains that have not agreed cannot be treated as reliable evidence of the full posterior uncertainty. Another mistake is to conclude that a low Rhat proves success. Chains can agree on the same incompletely explored region, especially with unrepresentative initialization or an inadequately long run.

Diagnostics also cannot distinguish weak identification from every computational problem, establish that the joint model is correctly specified, or replace scientific reasoning about the priors and data. If chains disagree, inspect traces and sampler warnings, assess whether the result is sensitive to initialization or parameterization, and rerun as needed. If chains agree but ESS or Monte Carlo error is inadequate, more effective sampling may be needed. If diagnostics are satisfactory but the posterior remains broad, that may reflect genuine limited information from the sparse events; do not report false precision to compensate.