Rank-Normalized R-Hat For Multiple Chains in Bayesian Trials

Why Compare Multiple Chains?

A trace plot shows how a parameter’s sampled values change across iterations. Seeing several traces overlap and move through a similar range is reassuring, but visual inspection alone can miss differences in chain location or spread. R-hat summarizes one such comparison: it asks whether the variation between chains is small relative to the variation within chains.

Rank-normalized split R-hat is a diagnostic for a scalar quantity of interest, such as a treatment effect, a baseline risk, or a derived clinical-trial estimand. It uses multiple chains targeting the same posterior distribution. The procedure splits each chain, converts pooled draws to normal scores based on their ranks, and compares within-chain and between-chain variation on that transformed scale. A folded version also checks for differences in spread that may not show up as differences in location.

Here \(W\) summarizes variation within chains, while \(B\) reflects how far the chain means are spread from their overall mean. If the chains explore the same target distribution, their means should not remain systematically separated, and the two variance estimates should be similar. A value of \(\widehat{R}\) near one is consistent with that expectation. It is a diagnostic signal, not a proof that the chains have converged or that the model is correct.

Constructing Rank-Normalized Split R-hat

Start with multiple chains that use the same model and target posterior distribution. After warmup has been removed, split each retained chain into two halves, usually its first and last halves. If the length is odd, one draw can be omitted so the halves have equal length. Splitting makes a chain’s early and late behavior contribute separately: a chain that drifts or changes location over time may otherwise have a single overall mean that conceals the problem.

Pool the draws from all \(2m\) split chains and assign each draw a rank \(r\) among the \(S=2mn\) pooled draws. For ties, use their average rank. Convert each rank to a normal score using the inverse standard normal cumulative distribution function \(\Phi^{-1}\):

This rank transformation, often called the Blom transformation, replaces the original values with scores determined by their relative order. Calculate \(W\), \(B\), \(\widehat{V}\), and \(\widehat{R}\) for these scores. Because the comparison is based on ranks, this version is less dependent on the original parameter’s scale and is more robust to some skewed or heavy-tailed marginal distributions than an R-hat calculated directly on raw values.

Rank normalization mainly checks for differences in location. To make a complementary check for differences in scale, calculate the absolute deviations of the original draws from their pooled median, rank-normalize those deviations by the same procedure, and calculate another split R-hat. This is the folded rank-normalized R-hat. Report the larger of the rank-normalized and folded values as rank-normalized split R-hat. A high folded value can flag chains with similar centers but different spreads.

Worked Calculations

Assumptions and Interpretation

This diagnostic is most informative when the chains are independent runs with dispersed initial values, use the same model and posterior target, and are assessed after warmup. The retained portions should be intended to sample from a stationary distribution. If chains target different distributions because of different code, data, or model settings, their R-hat comparison does not answer the intended convergence question. Nor does a favorable R-hat repair a misspecified model or an incorrect likelihood.

In practice, values close to one are reassuring only in conjunction with trace plots and other checks. A threshold such as 1.01 is sometimes used as a stringent warning rule, but it is not a universal guarantee: interpretation depends on the analysis, parameter, and consequences of inadequate computation. A value appreciably above one calls for investigation. A value slightly below one can occur from finite-sample variation and is not evidence of “better than converged” sampling.

Check the diagnostic for parameters that matter to the trial conclusions, including clinically interpretable derived quantities where appropriate. A single model parameter can look satisfactory while a nonlinear function of several parameters, such as a predicted risk difference, behaves differently across chains. The trace plot helps identify patterns R-hat compresses into one number: drift, long periods of little movement, separated regions, or changes in spread.

Common Pitfalls and Alternatives

• Calling a value near one proof of convergence. R-hat compares chains on a particular quantity; it cannot establish that every relevant posterior feature has been explored. Use it with traces and other diagnostics.
• Using only the ordinary, unsplit statistic. A whole-chain summary can obscure a difference between early and late behavior. Splitting makes that kind of within-chain change more visible.
• Ignoring the folded value. Chains can have similar centers but different spreads. Check the folded diagnostic as well as the rank-based location diagnostic.
• Comparing chains that do not share a target. Different model specifications or data inputs undermine the interpretation of the comparison. Confirm that chains were run under the same posterior model.
• Reading R-hat as a clinical effect measure. It is a dimensionless computational diagnostic, not an estimate of treatment benefit and not an interval in clinical units.

A raw-scale split R-hat is a simpler alternative, but it can be more sensitive to scale and distributional shape. Rank normalization improves the comparison for many non-normal posterior distributions, and folding adds sensitivity to scale differences. Trace plots provide a visual, time-ordered complement; unlike R-hat, they can reveal the pattern of a problem, but they do not reduce that evidence to a numerical summary. No one of these checks is sufficient by itself.

R-hat also has limitations. A value near one can occur when all chains become stuck in the same region, or when chains fail to visit a relevant low-probability region. It is a comparison across chains, not a direct measure of how precisely a posterior expectation or quantile has been estimated. It does not replace checks tailored to particular sampling methods or parameter constraints.

A Short Justification of the Variance Comparison

The construction combines a within-chain estimate with a between-chain estimate. The weight on \(W\) is \((n-1)/n\), and the weight on \(B\) is \(1/n\), so \(\widehat{V}\) lies between these components when both are nonnegative. If \(B=W\), then:

If \(B>W\), then \(\widehat{V}>W\), so \(\widehat{R}>1\). This explains why persistent separation among chain means tends to raise the statistic. The calculation is a variance comparison, however, not a theorem that \(\widehat{R}=1\) proves stationarity or complete exploration. The split, rank, and folded steps improve the diagnostic’s ability to reveal certain discrepancies; they do not eliminate its limitations.