Why HMC Warnings Need Diagnosis
Hamiltonian Monte Carlo (HMC) can explore complex posterior distributions efficiently, but its numerical trajectories may struggle with sharply changing curvature, poorly scaled parameters, or difficult posterior dependence. A warning is evidence about a particular aspect of sampling; it is not, by itself, a diagnosis of the cause. For Bayesian nonlinear mixed-effects (NLME) models, the challenge is to connect the pattern of warnings to the posterior geometry and then test a remedy that addresses that pattern.
The main signals considered here are divergent transitions, low effective sample size (ESS), high rank-normalized Rhat, and transitions that reach the maximum tree depth. They are related but not interchangeable. For example, increasing the number of post-warmup draws may improve Monte Carlo precision when chains are already exploring the same posterior, but it cannot reliably repair chains that are exploring different regions or missing a difficult region altogether.
Read Each Signal for What It Measures
A divergent transition occurs when the numerical trajectory departs too far from the intended Hamiltonian dynamics, often in a region of high or rapidly changing curvature. Divergences are especially important because they can cluster in a part of the posterior that the sampler then represents poorly. A small divergence fraction is not automatically harmless: inspect where divergences occur in parameter space and whether they are associated with substantively important quantities.
A low effective sample size means that autocorrelation has reduced the information in the recorded draws relative to the same number of independent draws. For \(M\) draws and autocorrelations \(\rho_t\) at lag \(t\), a common approximation is \(ESS \approx M/(1+2\sum_{t=1}^{\infty}\rho_t)\). ESS may be low for one parameter or for a particular posterior tail even when other summaries appear stable. Monte Carlo error, rather than a single universal ESS cutoff, helps determine whether the available draws are adequate for a planned summary.
Rank-normalized Rhat compares within-chain and between-chain variation after accounting for the ranks of draws. Values appreciably above 1 indicate that chains may not yet be sampling the same stationary distribution. A value near 1 is reassuring but does not prove that the model is correct or that every posterior region has been explored. An Rhat around 1.01 is often used as a stringent screening target, not as a guarantee or a universal pass/fail boundary.
A maximum tree-depth warning means that a NUTS trajectory reached the configured depth limit before meeting its stopping condition. This can indicate that long trajectories are needed to move effectively, but it is not equivalent to a divergence. Raising the limit may permit longer trajectories, with added computational cost; first check whether there is a more fundamental geometry or mixing problem.
Why Curvature and Scaling Matter
HMC uses a numerical integrator, commonly leapfrog integration, to approximate a trajectory through the posterior. The step size must be small enough to represent the local geometry. In a region where curvature changes sharply, a step size that works elsewhere may make the numerical energy error too large. A divergence is the sampler's indication that this error crossed a threshold. Reducing the step size can improve numerical resolution, but if the posterior has a narrow funnel or extreme parameter scaling, the sampler may need impractically small steps throughout the trajectory.
A short justification follows from the behavior of a numerical approximation. If a trajectory has \(L\) integration steps of size \(\epsilon\), reducing \(\epsilon\) generally reduces integration error per step, though it also increases the number of steps needed to cover a trajectory of similar length. Consequently, a smaller step size can suppress integration failures without changing the posterior target, but it may increase computation and will not necessarily resolve poor parameter coordinates or weakly identified dependence.
For a hierarchical parameter, a centered representation might write \(b_i \sim N(0,\tau^2)\), where \(b_i\) is a subject-level effect and \(\tau\) its population standard deviation. A non-centered representation writes \(\eta_i \sim N(0,1)\) and \(b_i=\tau\eta_i\). With weak individual information and small \(\tau\), the centered form can create a funnel-shaped geometry; the non-centered form may be easier to explore. When subject-level data are highly informative, the centered form can instead be more efficient. Reparameterization changes coordinates, not the intended scientific model, provided the transformation and its probability density are handled correctly.
Worked Example: Responding to Divergences
Suppose four chains each contain 1,000 post-warmup draws for a hypothetical NLME model. The sampler reports 27 divergent transitions among the 4,000 draws. Thus the observed fraction is \(27/4000=0.00675\), or \(0.675\%\). This fraction describes the runs; it does not establish that the remaining draws represent every important posterior region adequately.
Assume the divergences occur mostly when the estimated between-subject standard deviation \(\tau\) is near zero, and the chains otherwise have Rhat values near 1. This pattern is consistent with difficult geometry near a variance-component boundary, but it does not prove that geometry is the only problem. Plot divergent and non-divergent draws against \(\tau\) and related subject effects, and check whether the problematic region affects the target summaries.
A targeted next run could use a non-centered representation \(b_i=\tau\eta_i\), with \(\eta_i \sim N(0,1)\), if the data provide limited information about individual effects. Alternatively, if diagnostics suggest the model is otherwise well parameterized and the integrator is simply too coarse, a higher target acceptance probability can lead adaptation to select a smaller step size. After either change, rerun warmup and sampling, then check divergence locations, ESS, Rhat, and posterior summaries. The remedy is not established merely because the divergence count falls: the chains must still explore the intended posterior, and the model and prior must remain scientifically appropriate.
Worked Troubleshooting Examples
Worked Example: Low ESS and High Rhat Across Chains
Consider a hypothetical fit with four chains and 1,000 post-warmup draws per chain for a log-clearance parameter. There are \(M=4\times1000=4000\) draws. Suppose the estimated sum of positive-lag autocorrelations used in the ESS approximation is 19.5. Then
Only about 100 independent-draw equivalents are available for this parameter under the approximation, despite the 4,000 recorded draws. Suppose rank-normalized Rhat is 1.08 and trace plots show that one chain remains at higher log-clearance values than the other three. The high Rhat and chain separation indicate that simply extending the post-warmup run is not the first response: the chains may not yet be exploring the same distribution.
Check initial values, warmup behavior, posterior dependence with other parameters, and whether the parameter is on a scale that produces extreme curvature. If log-clearance and another parameter have very different numerical scales, rescale the parameterization while preserving the model. If a hierarchical dependence is implicated, compare centered and non-centered forms, guided by how informative the individual data are. A longer warmup is appropriate if adaptation was evidently unstable or too brief, but it should be followed by fresh post-warmup diagnostics. The goal is not just to obtain a smaller Rhat; it is to obtain chains with overlapping exploration and adequate ESS for the summaries of interest.
Worked Example: Interpreting a Tree-Depth Warning
In a hypothetical run, 42 of 4,000 post-warmup transitions reach the maximum tree depth of 12. The fraction is \(42/4000=0.0105\), or \(1.05\%\). Suppose there are no divergences, Rhat is 1.003 for the reported parameters, and bulk ESS is adequate for the planned posterior means. These diagnostics are more reassuring than if tree-depth warnings occurred alongside divergences and poor chain agreement, but they do not make the warning irrelevant.
First inspect which parameters and regions are involved, and whether the affected quantities have low tail ESS or unstable interval estimates. If estimates are adequate and the longer trajectories are simply costly, increasing the maximum depth may add computation without meaningful benefit. If transitions repeatedly need longer trajectories and mixing is poor, test whether parameter rescaling or a more suitable parameterization reduces the need for long trajectories. Only after these checks should a higher maximum depth be considered. If the limit is raised, compare ESS per unit of computation as well as the warning count; a larger limit is not itself evidence of better sampling.
A Practical Sequence for Troubleshooting
Change one main feature at a time where feasible. That makes it easier to identify what improved sampling, while retaining the model’s scientific meaning. A useful sequence is:
Record which chains, parameters, and posterior regions are involved. Compare trace plots and pairwise relationships rather than treating the total warning count as a complete diagnosis.
Look for parameters with very different numerical magnitudes, strong posterior dependence, boundary behavior, or a hierarchical funnel. Consider a scale change or a centered/non-centered representation when the pattern supports it.
For divergences, investigate geometry and integration accuracy. For high Rhat, investigate chain disagreement and initialization. For low ESS, investigate autocorrelation and parameter dependence. For tree-depth warnings, weigh trajectory needs against cost and other diagnostics.
After a change, use adequate warmup and examine the same diagnostics, including whether posterior summaries are stable and scientifically interpretable. If warnings persist, revisit the model geometry rather than escalating settings without limit.
Choosing Among Remedies
| Observed pattern | First checks | Potential targeted remedy | Important caution |
|---|---|---|---|
| Divergences concentrated in a narrow region | Map divergent draws against parameters and subject effects; inspect curvature and boundaries | Reparameterize; rescale; consider a smaller step size through adaptation | Fewer divergences do not prove the difficult region is adequately explored |
| High Rhat or separated traces | Compare chains, initialization, warmup behavior, and posterior dependence | Resolve geometry or initialization issues; use longer warmup if adaptation was insufficient | More post-warmup draws do not automatically reconcile chains targeting different regions |
| Low ESS with chains that overlap | Inspect autocorrelation, parameter dependence, and the summary-specific ESS | Improve parameterization or scaling; collect more draws if exploration is otherwise sound | More draws reduce Monte Carlo error but may be inefficient if mixing remains poor |
| Maximum tree-depth warnings | Check their frequency, affected quantities, divergences, ESS, and computational cost | Address geometry first; consider a higher limit if longer trajectories are useful | Raising the limit can increase computation and does not cure a poor posterior geometry |
Prior changes require particular care. A stronger prior can regularize a weakly informed parameter and sometimes improve geometry, but it also changes the posterior distribution. Use stronger prior information only when it is scientifically justified; examine prior sensitivity and report the resulting assumptions. Do not tighten a prior solely to make a warning disappear.
Common Pitfalls and Limits
- Treating zero divergences as proof of convergence. A run without divergences can still have high Rhat, low ESS, or unexplored posterior regions. Check the diagnostics together.
- Increasing the tree-depth limit as a universal fix. This can permit longer trajectories, but it may only make a poorly scaled or strongly dependent model more expensive to sample.
- Adding draws before checking chain agreement. More draws can improve precision after adequate exploration; they do not guarantee that chains trapped in different regions will meet.
- Changing priors to silence warnings. A prior is part of the statistical model. Its influence must be scientifically defensible, not merely computationally convenient.
- Assuming one parameterization is always best. Non-centering can help when group-level scales are weakly informed; centering may be more efficient when the data strongly inform individual effects. Compare performance and retain the interpretation of the model.
- Equating a warning with model inadequacy. Sampling diagnostics concern computation and exploration, not whether the likelihood or scientific assumptions describe the data well. Good sampling does not establish model adequacy.
These remedies are diagnostic experiments, not guarantees. A transformation that improves one parameter’s ESS can worsen another’s; a smaller step size can reduce divergences while increasing runtime; and more warmup helps only when adaptation has not adequately settled. When a change materially shifts posterior summaries, investigate whether improved exploration revealed previously missed posterior mass or whether the change altered the model, prior, or target distribution.
Check Your Understanding
Answer each question by connecting the diagnostic pattern to an appropriate next check or remedy.
- A model has several divergences near a small between-subject standard deviation. What should be examined before simply increasing the number of post-warmup draws?
- Four chains each have 500 post-warmup draws, and the ESS approximation uses a positive-lag autocorrelation sum of 9.5. Calculate the approximate ESS.
- Rhat is 1.06 and one trace remains separated from the others. Why is a longer post-warmup run alone not a sufficient remedy?
- A few transitions reach maximum tree depth, but there are no divergences and the target summaries have adequate ESS. What should guide the decision to raise the depth limit?
- Why might a stronger prior reduce a sampling difficulty, and why should it not be used merely to remove a warning?
Solutions. (1) Map divergent draws against the standard deviation and related effects, inspect the hierarchical geometry, and consider whether a centered or non-centered representation is appropriate before choosing a remedy. (2) There are \(M=4\times500=2000\) draws, so \(ESS\approx2000/[1+2(9.5)]=2000/20=100\). (3) The separated chain suggests chains may not be exploring the same stationary distribution; more draws can prolong disagreement rather than resolve its cause. Investigate initialization, warmup, and posterior geometry. (4) Check which parameters are affected, whether their summaries or tail ESS are inadequate, and whether longer trajectories justify the extra computational cost. (5) A stronger prior may regularize weakly informed directions, but it changes the posterior distribution and must be scientifically justified and assessed for sensitivity.