Reporting Bayesian NLME Sampling Diagnostics

Why Sampling Diagnostics Belong in the Report

A Bayesian NLME analysis reports posterior summaries, but those summaries are computed from a finite set of simulated draws. A reader therefore needs to know both what was estimated and whether the simulation appears to have explored the posterior adequately to estimate it. Sampling diagnostics support that assessment; they do not certify that a model is correct or that every part of its posterior has been explored.

A useful report makes the sampling procedure reproducible and connects diagnostics to the quantities being interpreted. State the number of chains, the warmup and retained draws per chain, important sampler settings, and diagnostics for relevant parameters and derived quantities. Keep per-chain counts distinct from totals: with three chains, for example, 500 warmup iterations per chain means 1,500 warmup iterations across the analysis.

The introductory material on HMC warmup and adaptation, chain initialization, trace plots, rank-normalized Rhat, effective sample size, and Monte Carlo standard errors establishes the diagnostic concepts used here. The reporting task is to give enough detail that readers can inspect those concepts in context, rather than presenting one reassuring statistic as a general guarantee.

What to Report

For reproducibility, identify the software and version, the model and parameterization used for sampling, and the sampler settings that materially affect the run. For HMC, these commonly include the target acceptance probability used during adaptation and the maximum tree depth. Report any changes from defaults, thinning if used, and any sampling warnings. Seeds can be useful for reproducibility, although they do not ensure identical results across software versions or computational environments.

Give chain and iteration counts explicitly. State the number of chains, warmup iterations per chain, retained iterations per chain, and—if useful—the corresponding totals. Also say whether the reported totals refer to attempted transitions or retained draws. In standard reporting, warmup draws are excluded from posterior summaries; do not combine warmup and retained counts without labeling them.

Report diagnostics for parameters that matter to the analysis, not just a selected parameter that behaved well. In an NLME model, this often includes population fixed effects, residual-error parameters, random-effect standard deviations and correlations, and clinically relevant derived quantities such as exposure or predicted concentrations. State whether the values shown are maxima or minima across parameters, or provide a parameter-level table. An overall maximum can quickly flag a concern, while the table shows where it occurs.

Rhat, Effective Sample Size, and Monte Carlo Error

Rank-normalized Rhat compares variation within chains with variation between chains, using split chains and rank-based transformations to improve sensitivity to some forms of nonconvergence. Values close to 1 indicate that the chains agree in the ways Rhat assesses. Report the maximum rank-normalized Rhat across the monitored quantities, and identify the quantities attaining it. A value below 1.01 is a common practical screening target, not a proof that chains have reached or fully explored the target posterior.

Effective sample size (ESS) describes how much independent-draw information the autocorrelated simulation provides for a particular posterior quantity. Report bulk ESS for central posterior summaries and tail ESS when tail quantiles or interval limits are important. ESS is not the literal number of draws retained: 4,500 retained, highly autocorrelated draws can contain substantially less information than 4,500 independent draws. There is no single ESS threshold that is adequate for every estimand; the required precision depends on the purpose of the analysis.

Monte Carlo standard error (MCSE) quantifies simulation error in a numerical posterior estimate, such as a posterior mean. It is distinct from posterior standard deviation, which describes posterior uncertainty about the parameter. For a posterior mean, the usual ESS-based approximation is the posterior standard deviation divided by the square root of the effective sample size.

This approximation follows from the variance of an average. If \(N\) draws were independent with variance \(\sigma_{\theta}^2\), the variance of their average would be \(\sigma_{\theta}^2/N\), so its standard error would be \(\sigma_{\theta}/\sqrt{N}\). For correlated draws, ESS is the corresponding information-equivalent number of independent draws. Replacing \(N\) by ESS and \(\sigma_{\theta}\) by the estimated posterior standard deviation gives the displayed approximation. It is an estimate of simulation error, not a confidence interval for the parameter.

For posterior quantiles, tail ESS and quantile-specific MCSE estimates are more relevant than bulk ESS alone. If quantile MCSEs are reported, identify the method or software used because their estimation is not generally the same calculation as the posterior-mean approximation above. Report enough precision to assess simulation error relative to the scientific precision needed; excessive decimal places do not make an estimate more certain.

Reporting Divergences and Other Sampler Warnings

An HMC divergence indicates that numerical integration had difficulty accurately exploring part of the posterior geometry. Report the number of divergent transitions after warmup, and state the denominator. Also report relevant warnings, such as transitions reaching the maximum tree depth. A count alone can obscure whether divergences are isolated or concentrated in a consequential region; where available, inspect diagnostic plots that show their location in parameter space.

Zero observed divergences is useful information, but it is not evidence that every part of the posterior has been adequately explored. Conversely, a divergence count is not itself a diagnosis of model inadequacy. It is a sampling warning that needs interpretation alongside chain behavior and the model’s posterior geometry. Increasing a sampler setting may reduce a warning, but the resulting run still needs to be assessed and reported rather than declared reliable solely because the warning disappeared.

A Practical Reporting Sequence

A concise report can follow the order below. This makes the numerical diagnostics interpretable without turning them into a checklist that substitutes for judgment.

Common Pitfalls and Limits

• Mixing per-chain and total counts. “1,500 warmup iterations” is ambiguous for three chains. State “500 per chain; 1,500 total” when that is what occurred.
• Treating Rhat near 1 as a certificate. Rhat assesses chain agreement, not whether the model is appropriate or all posterior regions were found. Interpret it with trace plots, ESS, and other diagnostics.
• Calling ESS the retained draw count. ESS adjusts for autocorrelation and is quantity-specific. Report retained draws and ESS separately.
• Confusing posterior uncertainty with simulation error. Posterior standard deviation describes uncertainty represented by the posterior; MCSE describes numerical uncertainty from using a finite simulation to estimate a posterior summary.
• Reporting only a divergence percentage. Include the count, denominator, and whether the count is after warmup. A small percentage does not guarantee that problematic transitions are irrelevant.
• Equating sampling reliability with model adequacy. Diagnostics concern the computation and its exploration of the specified posterior. They do not establish that the structural model, assumptions, or prior choices are scientifically appropriate.

The main alternative to a compact numerical summary is a fuller diagnostic presentation, including parameter-level tables and trace or divergence plots. A single maximum Rhat and minimum ESS are efficient for a brief report, but can hide which quantity is problematic; a table or supplement is preferable when many NLME parameters or derived quantities are central. Neither format replaces interpretation. Numerical summaries are compact and comparable, while plots can reveal patterns that an aggregate statistic conceals.

Diagnostics also have limits: they are estimated from finite simulations and can fail to detect some forms of poor exploration. Their practical meaning depends on the estimand. A run adequate for a posterior mean may not give sufficiently precise tail quantiles, and a parameter-level report may omit a derived prediction that drives the scientific conclusion. Report diagnostics for the quantities used to support the conclusions, and state any material remaining concerns.